Image sensing apparatus with exposure controller

ABSTRACT

An image sensing apparatus includes: an image sensor which generates an electrical signal commensurate with an amount of incident light, and has a photoelectric conversion characteristic comprised of a linear characteristic area where the electrical signal is outputted after being linearly converted in relation to the amount of incident light, and a logarithmic characteristic area where the electrical signal is outputted after being logarithmically converted in relation to the amount of incident light; an exposure evaluation value detector which detects an exposure evaluation value based on luminance information acquired from a subject in sensing an image of the subject; and an exposure controller which acquires a setting value for exposure based on the exposure evaluation value detected by the exposure evaluation value detector to control exposure of the image sensing apparatus, wherein the exposure controller determines a subject luminance for exposure setting based on the exposure evaluation value, and controls the exposure in such a manner that an output of the image sensor corresponding to the subject luminance for exposure setting is obtained from the linear characteristic area of the image sensor.

This application is a divisional application of and claims the benefitof priority from application Ser. No. 11/138,247 filed May 26, 2005,entitled “Image Sensing Apparatus,” and currently pending, which isbased on Japanese Patent Application No. 2004-160800 filed on May 31,2004, and No. 2005-77563 filed on Mar. 17, 2005, the contents of whichare hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image sensing apparatus providedwith an image sensor for generating an electrical signal commensuratewith the amount of incident light, and particularly relates to an imagesensing apparatus using an image sensor which has a photoelectricconversion characteristic comprised of a linear characteristic areawhere the electrical signal is outputted after being linearly convertedin relation to the amount of incident light, and a logarithmiccharacteristic area where the electrical signal is outputted after beinglogarithmically converted in relation to the amount of incident light,namely, an image sensor that is switchable between a linear operativestate and a log operative state.

2. Description of the Related Art

Heretofore, there has been known an image sensor (also called as “logsensor”) constructed such that a logarithm conversion circuit providedwith a MOSFET or a like device is added to a solid-state image sensingdevice comprised of photoelectric conversion elements such asphotodiodes arrayed in a matrix, wherein an output characteristic of thesolid-state image sensing device are converted in such a manner that anelectrical signal is logarithmically converted according to the amountof incident light by utilizing a subthreshold characteristic of theMOSFET. Among such image sensors, there is known an image sensor that isswitchable between a linear operative state in which an electricalsignal is outputted after being linearly converted according to theamount of incident light, and the aforementioned log operative state,according to the output characteristic inherent to the solid-state imagesensing device, namely, according to the amount of incident light.

For instance, Japanese Unexamined Patent Publication No. 2002-77733discloses an image sensing apparatus constructed such that the apparatusis automatically switchable from a linear operative state to a logoperative state by applying a specific reset voltage to a MOSFET, andthat the switching point of the linear operative state and the logoperative state is substantially identical to each other in all thepixels. Further, Japanese Unexamined Patent Publication No. 2002-300476discloses an image sensing apparatus constructed such that the apparatusis automatically switchable from a linear operative state to a logoperative state, and that the potential state of a MOSFET iscontrollable by controlling the reset time of the MOSFET.

The aforementioned image sensor has a merit that, in the linearoperative state thereof, a high contrast image signal is obtainable froma low luminance subject image because the output proportional to theamount of electric charge generated in the photoelectric conversionelements is obtained. However, the image sensor has a demerit that thedynamic range is narrow. On the other hand, in the log operative stateof the image sensor, although a wide dynamic range is secured becausethe output that has been natural-logarithmically converted according tothe amount of incident light is obtained, contrast becomes poor becausethe image signal is logarithmically compressed.

The image sensing apparatuses recited in the above publications merelydisclose that the image sensor is automatically switchable from thelinear operative state to the log operative state. In light of themerits and demerits of the linear operative state and the log operativestate, it is desirable to provide an image sensing apparatus which notonly enables to perform automatic switching but also enables to performa sensing operation by positively utilizing the merits of the linearoperative state and the log operative state. For instance, in automaticexposure control, controlling the exposure in association with thesubject luminance, and with the switching point from the linearoperative state to the log operative state enables to perform optimalautomatic exposure control, utilizing the merits of the linear operativestate and the log operative state.

SUMMARY OF THE INVENTION

In view of the problems residing in the prior art, it is an object ofthe present invention to provide an image sensing apparatus that enablesto capture a subject in an optimal exposure state, with a certaindynamic range being secured, commensurate with the amount of light fromthe subject, namely, according to a subject luminance by correlatingexposure control of the image sensing apparatus with a photoelectricconversion characteristic of an image sensor in the image sensingapparatus.

One aspect of the invention is directed to an image sensing apparatuscomprising: an image sensor which generates an electrical signalcommensurate with an amount of incident light, and has a photoelectricconversion characteristic comprised of a linear characteristic areawhere the electrical signal is outputted after being linearly convertedin relation to the amount of incident light, and a logarithmiccharacteristic area where the electrical signal is outputted after beinglogarithmically converted in relation to the amount of incident light;an exposure evaluation value detector which detects an exposureevaluation value based on luminance information acquired from a subjectin sensing an image of the subject; and an exposure controller whichacquires a setting value for exposure based on the exposure evaluationvalue detected by the exposure evaluation value detector to controlexposure of the image sensing apparatus, wherein the exposure controllerdetermines a subject luminance for exposure setting based on theexposure evaluation value, and controls the exposure in such a mannerthat an output of the image sensor corresponding to the subjectluminance for exposure setting is obtained from the linearcharacteristic area of the image sensor.

These and other objects, features and advantages of the presentinvention will become more apparent upon reading of the followingdetailed description along with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A through 1C are illustrations showing an external appearance ofa digital camera to which an image sensing apparatus as a firstembodiment of the invention is applied, wherein FIG. 1A is a top planview, FIG. 1B is a front view, and FIG. 1C is a rear view.

FIG. 2 is a block diagram of an image sensing process to be implementedby the digital camera.

FIG. 3 is an illustration showing an example of a color filter format ofan image sensor used in the digital camera.

FIG. 4 is a functional block diagram for explaining functions of a maincontroller equipped in the digital camera.

FIG. 5 is a flowchart showing an example of an overall operation of thedigital camera.

FIG. 6 is a chart for explaining definitions of terms relating toexposure control.

FIG. 7 is a schematic illustration of a two-dimensional MOS solid-stateimage sensing device, as an example of the image sensor.

FIG. 8 is a circuitry diagram showing an exemplified arrangement of eachpixel constituting the image sensor shown in FIG. 7.

FIG. 9 is an exemplified timing chart concerning a sensing operation ofthe image sensor.

FIG. 10 is a graph showing a photoelectric conversion characteristic ofthe image sensor.

FIG. 11 is a graph for explaining how the photoelectric conversioncharacteristic of the image sensor is changed.

FIG. 12 is a flowchart showing an operation example of detectingevaluation values concerning a subject image such as AE evaluationvalues based on a still image actually captured by the image sensor.

FIG. 13 is a flowchart showing an operation example of detectingevaluation values concerning the subject image such as AE evaluationvalues based on a moving image continuously captured by the imagesensor.

FIG. 14 is a functional block diagram for explaining functions of anevaluation value detector.

FIG. 15 is a diagram showing how an image sensing area to be metered isdivided into blocks according to multi-pattern metering by the imagesensor.

FIGS. 16A and 16B are graphs showing examples of luminance histograms bythe multi-pattern metering, wherein FIG. 16A is a main subject entireluminance histogram, and FIG. 16B is a peripheral subject entireluminance histogram.

FIG. 17 is a graph showing an example of a subject image entireluminance histogram when the output of the image sensor is saturated.

FIGS. 18A and 18B are graphs showing how the photoelectric conversioncharacteristic of the image sensor is changed in performing AE controlby exposure amount control, wherein FIG. 18A shows a case that theexposure amount is controlled to sense the subject image in a linearcharacteristic area if the subject luminance for exposure setting basedon AE evaluation values is located in a logarithmic characteristic area,and FIG. 18B shows a case that the exposure amount is controlled tosense the subject image in a relatively high output level area of alinear characteristic area if the subject luminance for exposure settingbased on AE evaluation values is located in a relatively low outputlevel area of the linear characteristic area.

FIG. 19 is a graph for explaining how the photoelectric conversioncharacteristic of the image sensor is changed if AE control is performedby dynamic range control.

FIG. 20 is a graph for explaining a linear conversion process incalculating an exposure amount control parameter.

FIG. 21 is a flowchart showing an example of a flow of calculating theexposure amount control parameter.

FIG. 22 is a graph for explaining a process in calculating the exposureamount control parameter.

FIGS. 23A and 23B are graphs each for explaining a process forcalculating the position of an inflection point of a photoelectricconversion characteristic in calculating a dynamic range controlparameter, wherein FIG. 23A shows a case that the photoelectricconversion characteristic is changed to achieve a predetermined sensoroutput corresponding to the luminance Lt1, and FIG. 23B shows a casethat the photoelectric conversion characteristic is modeled.

FIG. 24 is a functional block diagram for explaining functions of a maincontroller equipped in a digital camera as a second embodiment of theinvention.

FIG. 25 is a circuitry diagram showing an exemplified arrangement ofeach pixel constituting an image sensor of the digital camera shown inFIG. 24.

FIGS. 26A and 26B are examples of timing charts concerning an imagesensing operation of each pixel constituting the image sensor shown inFIG. 25, wherein FIG. 26A is a timing chart concerning a chargeaccumulating operation or an exposing operation in a vertical blankperiod of all the pixels, and FIG. 26B is a timing chart concerningelectric charge sweeping operation of pixels in each row by verticalscanning in a horizontal blank period after termination of the chargeaccumulation.

FIG. 27 is a graph showing how the photoelectric conversioncharacteristic of the image sensor is changed in performingaperture-control-based exposure amount control [A].

FIG. 28 is a graph showing how the photoelectric conversioncharacteristic of the image sensor is changed in performingexposure-time-control-based exposure amount control [C].

FIG. 29 is a graph showing how the photoelectric conversioncharacteristic of the image sensor is changed in performingphotoelectric-conversion-characteristic-control-based dynamic rangecontrol [B].

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Brief Description onEmbodiments

First, preferred embodiments of the invention are described briefly.

(1) An image sensing apparatus according to an aspect of the inventionis directed to an image sensing apparatus comprising: an image sensorwhich generates an electrical signal commensurate with an amount ofincident light, and has a photoelectric conversion characteristiccomprised of a linear characteristic area where the electrical signal isoutputted after being linearly converted in relation to the amount ofincident light, and a logarithmic characteristic area where theelectrical signal is outputted after being logarithmically converted inrelation to the amount of incident light; an exposure evaluation valuedetector which detects an exposure evaluation value based on luminanceinformation acquired from a subject in sensing an image of the subject;and an exposure controller which acquires a setting value for exposurebased on the exposure evaluation value detected by the exposureevaluation value detector to control exposure of the image sensingapparatus, wherein the exposure controller determines a subjectluminance for exposure setting based on the exposure evaluation value,and controls the exposure in such a manner that an output of the imagesensor corresponding to the subject luminance for exposure setting isobtained from the linear characteristic area of the image sensor.

In the arrangement (1), the exposure controller controls the exposure insuch a manner that the subject luminance for exposure setting, namely,the output of the image sensor corresponding to the target subjectluminance is obtained from the linear characteristic area of the imagesensor. Sensing a subject image under the above exposure control enablesto acquire a high contrast image signal by utilizing the feature of thelinear characteristic area, namely, a feature of a linear conversionoperation. Further, it is possible to obtain a wide dynamic range imagesignal, namely, a log-compressed image signal from a portion of asubject luminance which has not been used for exposure setting, namely,an area having a substantially higher luminance than the target area, byutilizing the feature of the logarithmic characteristic area, namely, afeature of a log conversion operation.

Here, the definition of the term “exposure control (hereinafter, alsocalled as “AE control”) is described referring to FIG. 6. In imagesensing apparatuses such as digital cameras and digital movie cameras,unlike so-called silver halide cameras, there are two factors in AEcontrol: one is a control in association with a photoelectric conversioncharacteristic of an image sensor, namely, a control by intentionallychanging the photoelectric conversion characteristic; and the other is acontrol on the total amount of light that reaches the image sensingplane of the image sensor, and the integration time of photocurrentafter photoelectric conversion. Throughout the specification and theclaims, the former is called as “dynamic range control”, and the latteris called as “exposure amount control”. The dynamic range control isexecuted, for instance, by controlling the switching point (hereinafter,called as “inflection point”) of the linear characteristic area and thelogarithmic characteristic area of the image sensor. Further, theexposure amount control is executed by controlling the aperture amountof a diaphragm, controlling the shutter speed of a mechanical shutter,or controlling the integration time of electric charge by control ofresetting operation to the image sensor.

According to the arrangement (1), a target image signal representing thesubject is constantly acquired from the linear characteristic area ofthe image sensor, and also, a predetermined dynamic range is secured byutilizing the feature of the logarithmic characteristic area. Forinstance, even if the overall subject luminance is relatively low, ahigh contrast image signal is obtained by utilizing the feature of thelinear characteristic area, and also, a dynamic range of a highluminance area is secured from the logarithmic characteristic area. Withthis arrangement, an optimal sensing output and video outputcommensurate with the amount of light from the subject are obtained fromthe image sensor.

(2) In the arrangement (1), preferably, the exposure evaluation value isdetected, by the exposure evaluation value detector, each from a mainsubject image area, and a peripheral subject image area of an imagesensing area of the image sensor, the image sensing area being comprisedat least of the main subject image area, and the peripheral subjectimage area located in a periphery of the main subject image area, andthe subject luminance for exposure setting is selected from the exposureevaluation value detected in the main subject image area.

In the embodiments of the invention, there is no constraint on a processfor acquiring the exposure evaluation value (hereinafter, also called as“AE evaluation value”). It is possible to meter the subject luminancewith use of a metering device or an equivalent element equipped in animage sensing device, or to detect an exposure evaluation value based onan image signal derived from an image actually captured by the imagesensor in light of simplifying the mechanism of the image sensingapparatus. In any case, a high contrast image signal utilizing thefeature of the linear characteristic area is obtained concerning themain subject by dividing the image sensing area into the main subjectimage area and the peripheral subject image area, and by selecting thesubject luminance for exposure setting based on the exposure evaluationvalue detected in the main subject image area.

According to the arrangement (2), a high contrast image signal utilizingthe feature of the linear characteristic area is obtained regarding amain subject such as an individual whose image is captured. Thisarrangement enables to obtain a high-quality image with high gradationperformance from the image sensor.

(3) In the arrangement (1) or (2), preferably, the exposure controllerincludes a photoelectric conversion characteristic information storagewhich stores the photoelectric conversion characteristic of the imagesensor acquired at the time of detecting the exposure evaluation valueby the exposure evaluation value detector.

In the arrangement (3), the photoelectric conversion characteristicinformation storage stores the photoelectric conversion characteristicor dynamic range information of the image sensor acquired at the time ofdetecting the exposure evaluation value. With this arrangement, a moreaccurate control parameter, as compared with the conventionalarrangement, can be obtained by referring to the dynamic rangeinformation in calculating the AE control parameter based on thedetected AE evaluation value.

According to the arrangement (3), since the AE control parameter can becalculated by referring to the dynamic range information of the imagesensor acquired at the time of detecting the AE evaluation value, and amore accurate control parameter can be obtained, optimal AE control canbe securely executed.

(4) In any of the arrangements (1) through (3), preferably, the exposurecontroller includes an exposure amount controller which controls anexposure amount to the image sensor, and the exposure amount controllerperforms exposure amount control in such a manner that the output of theimage sensor corresponding to the subject luminance for exposure settingis obtained from the linear characteristic area of the image sensor.

In the arrangement (4), the output of the image sensor corresponding tothe target subject luminance is obtained from the linear characteristicarea of the image sensor by control of the aperture value or control ofthe integration time. For instance, if the detected AE evaluation valueindicates that the subject is relatively bright, the exposure amountcontroller performs exposure amount control such as reducing of theaperture area or shortening of the integration time to obtain the outputof the image sensor corresponding to the subject luminance from thelinear characteristic area, so that the sensor output of the imagesensor corresponding to the subject luminance is not outputted from thelogarithmic characteristic area.

According to the arrangement (4), the AE control is executed in such amanner that the output of the image sensor corresponding to the targetsubject luminance is obtained from the linear characteristic area of theimage sensor by control of the aperture value or control of theintegration time. This arrangement enables to achieve the AE controlaccording to the embodiments of the invention by optimally controllingthe aperture value, the shutter speed, or the integration time of theimage sensor, without involving complexity in mechanical arrangement orcontrol program of the image sensor.

(5) In the arrangement (4), preferably, the exposure amount controllerincludes a photoelectric conversion characteristic information storagewhich stores the photoelectric conversion characteristic of the imagesensor acquired at the time of detecting the exposure evaluation valuedetected by the exposure evaluation value detector, and an exposureamount control parameter calculator which calculates a control parameterfor optimizing the exposure amount, and the exposure amount controlparameter calculator calculates an exposure amount control parameterbased on the exposure evaluation value detected by the exposureevaluation value detector, and the photoelectric conversioncharacteristic stored in the photoelectric conversion characteristicinformation storage.

In the arrangement (5), the photoelectric conversion characteristicinformation storage stores the dynamic range information of the imagesensor acquired at the time of detecting the exposure evaluation value.With this arrangement, a more accurate exposure amount controlparameter, as compared with the conventional arrangement, can beobtained by referring to the dynamic range information in calculatingthe exposure amount control parameter based on the detected AEevaluation value.

According to the arrangement (5), a more accurate exposure amountcontrol parameter can be obtained by referring to the dynamic rangeinformation stored in the photoelectric conversion characteristicstorage in calculating the exposure amount control parameter. Thisarrangement enables to securely perform AE control based on the optimalexposure amount control.

(6) In the arrangement (4) or (5), preferably, the exposure amountcontrol is performed in such a manner that the output of the imagesensor corresponding to the subject luminance for exposure setting isoutputted from a relatively high output level area in the linearcharacteristic area.

In the arrangement (6), for instance, if the detected AE evaluationvalue indicates that the subject is relatively dark, the exposure amountcontrol is performed in such a manner that the output of the imagesensor corresponding to the subject luminance is obtained from arelatively high output level area of the linear characteristic area.This arrangement enables to obtain a high contrast image even from a lowluminance subject image.

According to the arrangement (6), a high contrast image is obtained evenfrom a low luminance subject image by fully utilizing the linearcharacteristic area, which makes it possible to obtain a high-qualityimage.

(7) In any of the arrangements (1) through (3), preferably, the exposurecontroller includes a dynamic range controller which controls thephotoelectric conversion characteristic of the image sensor; and thedynamic range controller controls the photoelectric conversioncharacteristic of the image sensor in such a manner that the output ofthe image sensor corresponding to the subject luminance for exposuresetting is obtained from the linear characteristic area of the imagesensor.

In the arrangement (7), the output of the image sensor corresponding tothe target subject luminance can be obtained from the linearcharacteristic area of the image sensor by controlling the inflectionpoint of the linear characteristic area and the logarithmiccharacteristic area. For instance, if the detected AE evaluation valueindicates that the subject is relatively bright, the dynamic rangecontroller performs photoelectric conversion characteristic control suchas setting of the inflection point to the position corresponding to arelatively high output level of the image sensor to obtain the output ofthe image sensor corresponding to the subject luminance from the linearcharacteristic area, so that the sensor output of the image sensorcorresponding to the subject luminance is not outputted from thelogarithmic characteristic area.

According to the arrangement (7), the AE control is executed in such amanner that the output of the image sensor corresponding to the targetsubject luminance is obtained from the linear characteristic area of theimage sensor by controlling the photoelectric conversion characteristicof the image sensor. This arrangement enables to achieve the AE controlaccording to the embodiments of the invention by changing the operationstate of the image sensor by a bias voltage or the like, withoutinvolving complexity in mechanical arrangement or control program of theimage sensing apparatus.

(8) In the arrangement (7), preferably, the dynamic range controllerincludes a photoelectric conversion characteristic information storagewhich stores the photoelectric conversion characteristic of the imagesensor acquired at the time of detecting the exposure evaluation valueby the exposure evaluation value detector, and a dynamic range controlparameter calculator which calculates a control parameter for optimizingthe photoelectric conversion characteristic of the image sensoraccording to the subject luminance, and the dynamic range controlparameter calculator calculates a dynamic range control parameter basedon the exposure evaluation value detected by the exposure evaluationvalue detector, and the photoelectric conversion characteristic storedin the photoelectric conversion characteristic information storage.

In the arrangement (8), the photoelectric conversion characteristicinformation storage stores the dynamic range information of the imagesensor acquired at the time of detecting the exposure evaluation value.Accordingly, a more accurate dynamic range control parameter, ascompared with the conventional arrangement, can be obtained by referringto the dynamic range information in calculating the dynamic rangecontrol parameter based on the detected AE evaluation value.

According to the arrangement (8), a more accurate dynamic range controlparameter can be obtained by referring to the dynamic range informationstored in the photoelectric conversion characteristic informationstorage in calculating the exposure amount control parameter. Thisarrangement enables to securely perform AE control based on the optimaldynamic range.

(9) In the arrangement (4), preferably, the image sensor is configuredin such a manner as to execute photoelectric conversion in thelogarithmic characteristic area independently of an exposure time, theimage sensing apparatus further comprises a diaphragm, the exposureamount controller includes an aperture controller which controls theexposure amount based on an aperture setting value relating to controlof an aperture area of the diaphragm, and/or an exposure time controllerwhich controls the exposure amount based on an exposure time settingvalue relating to control of the exposure time to the image sensor, andthe exposure amount controller performs control of the exposure amountby the aperture controller and/or by the exposure time controller insuch a manner that the output of the image sensor corresponding to thesubject luminance for exposure setting is obtained from the linearcharacteristic area of the image sensor, and the aperture controller andthe exposure time controller are configured to control the exposureamount independently of each other.

In the arrangement (9), the aperture control and the exposure timecontrol are independently operative for exposure amount control, namely,exposure control. The aperture controller and/or the exposure timecontroller capable of obtaining respective setting values, namely,control parameters by changing the photoelectric conversioncharacteristic in accordance with the aperture control and/or theexposure time control, respectively, control the aperture value or theexposure time such as the integration time or the shutter opening time.Thereby, the output of the image sensor corresponding to the targetsubject luminance can be obtained from the linear characteristic area ofthe image sensor. For instance, if the detected AE evaluation valueindicates that the subject is relatively bright, exposure amount controlsuch as reducing of the aperture area or shortening of the exposure timeis performed to obtain the output of the image sensor corresponding tothe subject luminance from the linear characteristic area. Further, inthis arrangement, the exposure amount control for obtaining the outputof the image sensor corresponding to the subject luminance for exposuresetting from the linear characteristic area of the image sensor can beexecuted with use of the aperture control and/or the exposure timecontrol independently of each other. This arrangement enables toefficiently execute the exposure amount control with high latitudedepending on combination of the aperture control and the exposure timecontrol.

According to the arrangement (9), the output of the image sensorcorresponding to the target subject luminance is obtained from thelinear characteristic area of the image sensor by controlling theaperture value by the aperture controller, and/or by controlling theexposure time by the exposure time controller, wherein the aperturecontroller and the exposure time controller are capable of performingexposure amount control, namely, exposure control, independently of eachother. Further, the exposure amount control for obtaining the output ofthe image sensor corresponding to the subject luminance for exposuresetting from the linear characteristic area can be executed with use ofthe aperture control and/or the exposure time control independently ofeach other. This arrangement enables to efficiently execute the exposureamount control with high latitude depending on combination of theaperture control and the exposure control.

(10) In the arrangement (9), preferably, the exposure amount control isperformed in such a manner that the output of the image sensorcorresponding to the subject luminance for exposure setting is obtainedfrom a relatively high output level area in the linear characteristicarea of the image sensor.

In the arrangement (10), for instance, if the detected AE evaluationvalue indicates that the subject is relatively dark, the exposure amountcontrol by the aperture control and/or by the exposure time control isperformed to obtain the output of the image sensor corresponding to thesubject luminance for exposure setting from a relatively high outputlevel area of the linear characteristic area. This arrangement enablesto obtain a high contrast image even from a low luminance subject image.

According to the arrangement (10), the exposure amount control by theaperture control and/or by the exposure time control is performed insuch a manner that the output of the image sensor corresponding to thesubject luminance for exposure setting is obtained from a relativelyhigh output level area of the linear characteristic area. Thisarrangement enables to obtain a high contrast image even from a lowluminance subject image.

(11) In the arrangement (9) or (10), preferably, the exposure controllerincludes a dynamic range controller which controls the photoelectricconversion characteristic of the image sensor, and the dynamic rangecontroller controls the photoelectric conversion characteristic of theimage sensor in such a manner that the output of the image sensorcorresponding to the subject luminance for exposure setting is obtainedfrom the linear characteristic area of the image sensor.

In the arrangement (11), the output of the image sensor corresponding tothe target subject luminance can be obtained from the linearcharacteristic area of the image sensor by controlling the photoelectricconversion characteristic with use of the dynamic range controller,namely, by controlling the inflection point of the linear characteristicarea and the logarithmic characteristic area. For instance, if thedetected AE evaluation value indicates that the subject is relativelybright, the dynamic range controller performs photoelectric conversioncharacteristic control such as setting of the inflection point to theposition corresponding to a relatively high output level of the imagesensor to obtain the output of the image sensor corresponding to thesubject luminance from the linear characteristic area. Further, in thisarrangement, the exposure amount control for obtaining the output of theimage sensor corresponding to the subject luminance for exposure settingfrom the linear characteristic area can be executed by the dynamic rangecontrol, in addition to the aperture control and/or the exposure timecontrol independently of each other. This arrangement enables toefficiently execute the exposure amount control with high latitude, andin accordance with combination of the aperture control, the exposuretime control, and the dynamic range control.

According to the arrangement (11), the output of the image sensorcorresponding to the target subject luminance can be obtained from thelinear characteristic area of the image sensor by controlling of thephotoelectric conversion characteristic, namely, the inflection point,by the dynamic range controller. Further, in this arrangement, theexposure amount control for obtaining the output of the image sensorcorresponding to the subject luminance for exposure setting from thelinear characteristic area can be executed by the dynamic range control,in addition to the aperture control and/or the exposure time controlindependently of each other. This arrangement enables to efficientlyexecute the exposure amount control with high latitude depending oncombination of the aperture control, the exposure control, and thedynamic range control.

First Embodiment

In the following, a first embodiment of the invention is describedreferring to the drawings.

(Description on External Construction of Image Sensing Apparatus)

FIGS. 1A through 1C are diagrams showing external appearance of acompact digital camera 1 to which an image sensing apparatus as thefirst embodiment is applied, wherein FIG. 1A is a top plan view, FIG. 1Bis a front view, and FIG. 1C is a rear view. The digital camera 1, as anexample of image sensing apparatuses, has a power supply switch 101 anda release switch 102 on a top part of a camera body 10, a flash section103 and a taking lens aperture 104 on a front part thereof, and variousoperation buttons such as a mode setting switch 105, and a LCD section106 comprised of a liquid crystal display (LCD) monitor on a rear partthereof. A retractable lens barrel 20 is provided inside the camera body10, as well as various parts constituting the camera body 10.

The power supply switch 101 is a pressable type switch used to turn onand off the power source of the camera 1 to start and stop power supplyof the camera 1. Every time the power supply switch 101 is pressed, thepower source of the camera 1 is alternately and repeatedly turned on andoff. The mode setting switch 105 is adapted to set two modes, namely, astill image shooting mode of shooting a still image, and a moving imageshooting mode of shooting a moving image.

The release switch 102 is a pressable type switch, and is settable to ahalfway pressed state where the release switch 102 is pressed halfwaydown, and to a fully pressed state where the release switch 102 ispressed fully down. When the release switch 102 is pressed halfway downin the still image shooting mode, for example, a preparatory operationof shooting a still image of a subject such as automatic exposurecontrol and automatic focal adjustment, which will be described later,is executed. Subsequently, when the release switch 102 is pressed fullydown, an image shooting operation, namely, a series of operationscomprising exposing an image sensor, which will be described later,applying a predetermined image processing to an image signal acquired bythe exposure, and recording the processed signal in a memory card or alike device, are executed. On the other hand, when the release switch102 is pressed fully down in the moving image shooting mode, apredetermined moving image shooting operation is started. Subsequently,when the release switch 102 is pressed fully down again, the movingimage shooting operation is terminated.

While the release switch 102 is pressed halfway down in the still imageshooting mode, the flash section 103 flashes light to illuminate thesubject if the subject image is dark. The taking lens aperture 104 is anopening for guiding the subject light image to the retractable lensbarrel 20 provided inside the camera body 10. The LCD section 106 isadapted to playback or display an image recorded in a recording mediummounted in the camera body 10, or to display a through-image or alive-view image of the subject which has been video-shot in a shootingstandby period or in the moving image shooting mode. The camera body 10has a group of push switches such as a zoom switch, a menu selectionswitch, and a selection determination switch in addition to the modesetting switch 105.

The retractable lens barrel 20 constitutes a taking lens system whichguides the subject image through the taking lens aperture 104 to theimage sensor 30 arranged inside the camera body 10. The lens barrel 20is a lens barrel whose length is not changed even in zooming or focusingdriving. Namely, the lens barrel 20 does not protrude outside of thecamera body 10. Inside the lens barrel 20, there are provided a lensgroup 21 (see FIG. 2) constituting a taking optical system comprised ofa zoom lens block and a fixed lens block arrayed in series along anoptical axis, and a diaphragm 22 arranged at an appropriate position ofthe lens group 21. A shutter 23 is arranged at an appropriate positionof the lens group 21 to allow or block incidence of light along theoptical path of the taking optical system by opening/closing the shutter23. In other words, the exposure amount of the image sensor 30 iscontrolled based on the setting degree of the aperture area of thediaphragm 22, an opening/closing operation of the shutter 23, or otherfactor.

(Description on Entire Electrical Configuration of Image SensingApparatus)

FIG. 2 is a block diagram of an imaging process to be implemented by thedigital camera 1 as the first embodiment. The digital camera 1 has anoperating section 100, the retractable lens barrel 20, the image sensor30, a signal processing section 40, a main controller 50, and a drivingsection 60. The operating section 100 is constituted of the power supplyswitch 101, the release switch 102, and the mode setting switch 105.

The image sensor 30 photo-electrically converts the subject light imageformed through the lens group 21 in the lens barrel 20 into imagesignals of respective color components of red (R), green (G), and blue(B), namely, signals comprised of signal arrays representing pixelsignals received on the respective pixels of the image sensor 30according to the amount of light from the subject for outputting to thesignal processing section 40. In this embodiment, a log conversion typesolid-state image sensing device is used as the image sensor 30. Theimage sensor 30 is constructed such that output pixel signals, or outputelectrical signals generated by photoelectric conversion are outputtedin relation to the amount of incident light, after being logarithmicallyconverted. The image sensor 30 has such an output characteristic thatoutput pixel signals are outputted after being linearly converted whenthe amount of incident light is lower than a predetermined level. Theimage sensor 30 comprises a linear characteristic area where itsphotoelectric conversion characteristic is linear while the subjectimage is dark, and a logarithmic characteristic area where itsphotoelectric conversion characteristic is logarithmic while the subjectimage is bright. Further, the switching point, namely, the inflectionpoint of the linear characteristic area and the logarithmiccharacteristic area is controllable by a specific control signal,namely, a dynamic range control signal to be described later. Theconstruction, the operation and the like of the image sensor 30 will bedescribed later in detail.

A timing generating circuit or a timing generator 31 controls an imagesensing operation by the image sensor 30 such as electric chargeaccumulation based on exposure and readout of the accumulated electriccharge. The timing generating circuit 31 generates a predeterminedtiming pulse such as a pixel drive signal, a horizontal scanning signal,a vertical scanning signal, a horizontal scanning circuit drive signal,and a vertical scanning circuit drive signal based on a sensing controlsignal sent from the main controller 50, outputs the generated signalsto the image sensor 30, reads out a frame image every 1/30 second, forexample, in the moving image shooting mode, namely, in the through-imagedisplay mode, and outputs the signals successively to the signalprocessing section 40. Further, during exposure in the still imageshooting mode, the timing generating circuit 31 accumulates electriccharge in association with the exposure operation of the image sensor30, namely, photoelectrically converts the subject light image intoimage signals, and outputs the accumulated electric charge to the signalprocessing section 40. Further, the timing generating circuit 31generates a clock for analog-to-digital (A/D) conversion to be used inan ND converter 402, which will be described later.

The signal processing section 40 applies a predetermined analog signalprocessing and a predetermined digital signal processing to the imagesignals outputted from the image sensor 30. The image signal processingis implemented with respect to each of the pixel signals constitutingthe image signals. The signal processing section 40 includes an analogsignal processor 401, the A/D converter 402, a black level corrector403, a fixed pattern noise (FPN) corrector 404, an evaluation valuedetector 405, a white balance (WB) controller 406, a color interpolator407, a 3×3 color corrector 408, a gradation converter 409, a noisecanceller 410, and an image memory 411.

The analog signal processor 401 applies a predetermined analog signalprocessing to the image signals outputted from the image sensor 30,namely, an analog signal group representing light received on therespective pixels of the image sensor 30, and includes a correlationdouble sampling (CDS) circuit for reducing a reset noise included ineach analog image signal, and an auto gain control (AGC) circuit forcorrecting the level of the analog image signal. The AGC circuit has anamplifying function of amplifying the analog image signal with anadequate amplification ratio to compensate for insufficiency in signallevel of a captured image, in the case where adequate exposure was notobtained, so that the amplified analog signal level lies in the inputvoltage range of the A/D converter 402, which will be described later.

The A/D converter 402 has a function of converting the analog imagesignal outputted from the analog signal processor 401 into a digitalimage signal, namely, image data of 12 bits, for instance. The A/Dconverter 402 converts the analog image signal into a digital imagesignal based on the clock for A/D conversion sent from the timinggenerating circuit 31.

The black level corrector 403 implements computation: SD1-SD2 where SD1represents the level of the image signal outputted from the A/Dconverter 402, and SD2 represents the level of the image signal at adark time to correct the black level of the digital image signaloutputted from the A/D converter 402, namely, the image signal level atthe dark time, to a reference value e.g. 0 in terms of digital signallevel after A/D conversion. The black level correction is performedbased on dynamic range information of the image sensor 30 commensuratewith the photoelectric conversion characteristic of the image sensor 30outputted from the main controller 50. This is for the followingreasons. In the digital camera 1 according to the embodiment of theinvention, the photoelectric conversion characteristic of the imagesensor 30 is controllable, and the level of the digital image signaloutputted from the A/D converter 402 at the dark time is changed inresponse to change of the photoelectric conversion characteristic of theimage sensor 30. Thereby, accurate black level correction in accordancewith the change of the image signal level can be conducted.

The FPN corrector 404 removes a fixed pattern noise in the image signaloutputted from the black level corrector 403. The fixed pattern noise isa noise due to a variation among threshold values of field effecttransistors equipped in respective pixel circuits of the image sensor30, and results from a variation among output values of the pixelsignals generated by the respective pixels. The FPN corrector 404implements computation: SD3-SD4 where SD3 represents the level of theimage signal outputted from the black level corrector 403, and SD4represents the fixed pattern component of the image signal outputtedfrom the black level corrector 403.

The evaluation value detector 405 detects, based on the image signalactually acquired by the image sensing operation of the image sensor 30,evaluation values based on which automatic exposure (AE) control, autofocusing (AF) control, white balance (WB) control, or a like control isto be implemented, namely, based on AE evaluation values, AF evaluationvalues, white balance evaluation values (hereinafter, called as “WBevaluation values”), or the like. In case of conducting AE control,generally, the following steps are implemented:

(1) measuring the luminance level and the luminance range of a subjectas a target whose image is to be captured;

(2) calculating an exposure control amount necessary for securing anoutput from the image sensor commensurate with the luminance level andthe luminance range; and

(3) controlling the exposure amount and the like based on thecalculation result before actual shooting.

The evaluation value detector 405 calculates the luminance level and theluminance range of the subject based on the image signal actuallyacquired by the image sensor 30 to carry out the step (1), and outputsthem as AE evaluation values to the main controller 50, so that they canbe used for AE control, which will be described later.

In case of AF control, driving of the focus lens of the lens group 21along an optical axis direction, and an image sensing operation by theimage sensor 30 are alternately conducted to set the focus lens to sucha position that makes it possible to maximize the contrast of the imageacquired by the sensing operation, namely, a so-called hill-climbingsearch technique is adopted. The detected position of the focus lens isoutputted to the main controller 50 as an AF evaluation value, which, inturn, is used for AF control, which will be described later. Further,white balance control is implemented to correct the colors of the outputimage to those conforming to a light source color of the subject. Inthis embodiment, the luminance ratios and the luminance differences ofthe respective color components R, G, and B are calculated by theevaluation value detector 405 based on the image signal outputted fromthe FPN corrector 404, and the calculated luminance ratios and thecalculated luminance differences are outputted to the main controller 50as WB evaluation values. Exemplified methods for acquiring the AEevaluation values, the AF evaluation values, and the WB evaluationvalues will be described later in detail.

The white balance controller 406 conducts level conversion of the pixeldata of the respective color components R, G, and B, based on thedynamic range information of the image sensor 30, and the WB evaluationvalues outputted from the main controller 50, so that the image signalhas a predetermined color balance. In this embodiment, since the imagesensor 30 comprises a linear characteristic area and a logarithmiccharacteristic area, it is preferable to conduct white balancecorrection suitable for each of the linear characteristic area and thelogarithmic characteristic area by acquiring the WB evaluation valueswith respect to each of the linear characteristic area and thelogarithmic characteristic area.

The color interpolator 407 interpolates pixel data at the pixel positionwhere there is no color information in a frame image with respect toeach of the color components R, G, and B of the image signal outputtedfrom the white balance controller 406. Specifically, since a colorfilter format of the log conversion type image sensor 30 used in theembodiment adopts a so-called Bayer system in which green is arrayed ina checker pattern, and red and blue are each arrayed linearly, colorinformation is not sufficient. In view of this, the color interpolator407 interpolates pixel data at the pixel positions where there is noimage data, using a plurality of existing pixel data.

More specifically, regarding a frame image of the color component Ghaving pixels up to a high bandwidth, the color interpolator 407 masksimage data constituting the frame image with a predetermined filterpattern, and calculates an average of pixel data by excluding pixel datahaving a maximum value and a minimum value out of the pixel dataexisting around the target pixel position to be interpolated, with useof a median filter, and interpolates this average value as the pixeldata to be interpolated at the target pixel position. Regarding frameimages of the color components R and B, the color interpolator 407 masksimage data constituting each frame image with a predetermined filterpattern, calculates an average of pixel data existing in the vicinity ofthe pixel position, and implements interpolation by using the average aspixel data at the target pixel position.

FIG. 3 shows an example of the color filter format of the image sensor30. Image signals of the respective color components R, G, and B inrespective pixels are generated by the color interpolation with use ofthe color filter format, as shown below, for instance:

(i) Color interpolation equation for address 11 (B11):R11=(R00+R20+R02+R22)/4G11=(Gr10+Gb01+Gb21+Gr12)/4B11=B11

(ii) Color interpolation equation for address 12 (Gr12):R12=(R02+R22)/2G12=Gr12B12=(B11+B13)/2

(iii) Color interpolation equation for address 21 (Gb21):R21=(R20+R22)/2G21=Gb21B21=(B11+B31)/2

(iv) Color interpolation equation for address 22 (R22):R22=R22G22=(Gb21+Gr12+Gr32+Gb23)/4B22=(B11+B31+B13+B33)/4

The 3×3 color corrector 408 corrects the saturation or the tint of theimage signals of the respective color components R, G, and B outputtedfrom the color interpolator 407. The 3×3 color corrector 408 has threekinds of conversion coefficients with respect to each of the colorcomponents R, G, and B for converting the level ratio of image signalsof the color components R, G, and B, and corrects the saturation ofimage data by converting the level ratio with use of a conversioncoefficient conforming to a scene to be shot. For instance, the 3×3color corrector 408 linearly converts the image signals with use of nineconversion coefficients, namely, a1, a2, a3, b1, b2, b3, c1, c2, and c3,as follows:R′=a1·R+a2·G+a3·BG′=b1·R+b2·G+b3·BB′=c1·R+c2·G+c3·B

The gradation converter 409 non-linearly converts and offset-adjusts thelevel of the image signal with respect to each of the color componentsR, G, and B using a specified gamma characteristic, so that the imagesignals of the respective color components R, G, and B outputted fromthe 3×3 color corrector 408 attain appropriate output levels,respectively. Specifically, the gradation converter 409 corrects thegradation characteristic such as gamma curve and digital gain of theimage signals after the white balance adjustment and the colorcorrection to a gradation characteristic of the LCD section 106 or anexternally connected television monitor or the like. The gradationconverter 409 changes the gradation characteristic of the image signalbased on the dynamic range information outputted from the maincontroller 50, and the AE evaluation values and the like detected by theevaluation value detector 405.

The noise canceller 410 removes a noise component in the image signaloutputted from the gradation converter 409, and correctively acquiresdesired sharpness of the image by extracting/emphasizing an edgecomponent. The noise canceller 410 performs adequate correction bychanging a coring factor, which is a factor to be used in removing thenoise component in the image signal, extracting and emphasizing the edgecomponent based on the dynamic range information outputted from the maincontroller 50.

The image memory 411 includes a memory such as an ROM and an RAM, andtemporarily stores image data after the signal processing in the signalprocessing section 40. The image memory 411 has a capacity capable ofstoring image data corresponding to one frame, for instance.

A memory card interface (I/F) 412 is an interface for recording imagedata that has been generated in the signal processing section 40 in amemory card 107 for output. The memory card 107 is a memory in whichimage data such as a still image and a moving image is to be recordedfor storage. The memory card 107 is detachable from the digital camera 1to allow exchange of image data with an external recording medium. AnLCD display interface (I/F) 413 is an interface for converting the imagedata that has been generated in the signal processing section 40 for LCDdisplay into an image signal in compliance with NTSC standards or PALstandards, for instance, for outputting to the LCD section 106.

The main controller 50 comprises a central processing unit (CPU) and isadapted to centrally control shooting operation of the digital camera 1.Specifically, the main controller 50 controls operations of therespective elements of the signal processing section 40 based on theinformation sent from the respective elements of the signal processingsection 40 such as the AE evaluation values, the AF evaluation values,and the WB evaluation values, as well as on the operation mode of thedigital camera 1, by calculating and outputting operation informationsuch as parameters necessary for operating the respective elements ofthe signal processing section 40. Further, the main controller 50controls the timing generating circuit 31 for shooting operation,controls the driving section 60 for zooming and focusing driving of thelens group 21, and for driving of the diaphragm 22 and the shutter 23,and controls image signal outputting operation.

FIG. 4 is a functional block diagram for explaining functions of themain controller 50. The main controller 50 includes an informationreceiver 501, an information sender 502, a calculating section 510 witha memory unit 515, a control signal generating section 520, and aninput/output section 530.

The information receiver 501 acquires the AE evaluation values, the AFevaluation values, and the WB evaluation values which are detected bythe evaluation value detector 405 of the signal processing section 40,and distributes the respective evaluation values to correspondingparameter calculators provided in the calculating section 510. On theother hand, the information sender 502 reads out, from the memory unit515, the information necessary for the signal processing section 40 suchas the dynamic range information and the coring factor according toneeds, and distributes the information to the respective elements in thesignal processing section 40 according to needs.

The calculating section 510 calculates control parameters based on theevaluation values sent from the information receiver 501, and includesan AE control parameter calculating unit 5110 comprised of an exposureamount control parameter calculator 511, and a dynamic range controlparameter calculator 512, an AF control parameter calculator 513, awhite balance control parameter calculator 514, and the memory unit 515.

The memory unit 515 includes an ROM and an RAM, and is comprised of aphotoelectric conversion characteristic storage 516 for storing dynamicrange information of the image sensor 30, namely, a setting value of aphotoelectric conversion characteristic, a coring factor storage 517 forstoring a setting position of the coring factor to be used in the noisecanceller 410, and an LUT storage 518 for storing a lookup table (LUT)with which data acquired from the linear characteristic area and thelogarithmic characteristic area of the image sensor 30 areinterchangeably converted. The photoelectric conversion characteristicinformation storage 516 may store a photoelectric conversioncharacteristic itself, namely, a photoelectric conversion characteristiccurve as shown in FIG. 10, which will be described later. Further, theLUT storage 518 stores therein, other than the above lookup table,various lookup tables for data conversion such as a lookup table withwhich data is converted between the value of an exposure time and thevalue of an aperture area of the diaphragm, and the exposure timesetting value and the aperture setting value, a lookup table with whichdata is converted between the value of the inflection point, namely, theoutput level of the photoelectric conversion characteristic, and thephotoelectric conversion characteristic setting value, a lookup tablewith which data is converted between the number of saturated pixels, andthe value of change of the inflection point, a lookup table with whichthe photoelectric conversion characteristic setting value is outputtedbased on the maximum luminance level, and a lookup table with which achange of the photoelectric conversion characteristic setting value isoutputted based on a change of the maximum luminance level. Further, asdescribed above, the data stored in the photoelectric conversioninformation storage 516, the coring factor storage 517, and the LUTstorage 518 are sent to an appropriate processor in the signalprocessing section 40 from the information sender 502 according toneeds.

The AE control parameter calculating unit 5110 calculates a controlparameter for setting an optimal exposure amount for shooting and thephotoelectric conversion characteristic of the image sensor 30 toexecute exposure control or AE control commensurate with a subjectluminance. Specifically, the exposure amount control parametercalculator 511 of the AE control parameter calculating unit 5110calculates a control parameter for optimizing the exposure time and theaperture value, and calculates the exposure time setting value and theaperture setting value commensurate with the subject luminance based onthe AE evaluation values detected by the evaluation value detector 405,and the dynamic range information or the photoelectric conversioncharacteristic of the image sensor 30 at the time when the AE evaluationvalues stored in the photoelectric conversion characteristic informationstorage 516 were obtained.

The dynamic range control parameter calculator 512 calculates a controlparameter for optimizing the photoelectric conversion characteristic ofthe image sensor 30 commensurate with the subject luminance. The dynamicrange control parameter calculator 512 calculates the photoelectricconversion characteristic setting value based on which the subjectluminance used in setting the dynamic range of the image sensor 30 canattain a desired saturation output level of the image sensor 30. In sucha calculation, the dynamic range information of the image sensor 30 atthe time when the AE evaluation values stored in the photoelectricconversion characteristic information storage 516 were obtained isreferred to. The operation of the AE control parameter calculating unit5110 will be described later in detail.

The AF control parameter calculator 513 calculates a control parameterfor setting the optimal focal length in shooting a subject image, basedon the AF evaluation values detected by the evaluation value detector405. In calculating the control parameter, it is preferable to acquirethe AF evaluation values for reference from each of the logarithmiccharacteristic area and the linear characteristic area of the imagesensor 30 and to calculate a control parameter for rough metering,namely, for the AF evaluation values acquired from the logarithmiccharacteristic area, and a control parameter for precise metering,namely, for the AF evaluation value acquired from the linearcharacteristic area, respectively, by utilizing the features of therespective characteristic areas.

The white balance control parameter calculator 514 calculates a controlparameter for setting the color balance of the image signal to a desiredcolor balance based on the WB evaluation values detected by theevaluation value detector 405. In calculating the control parameter, itis preferable to acquire the WB evaluation values for reference fromeach of the logarithmic characteristic area and the linearcharacteristic area of the image sensor 30 and to calculate controlparameters suitable for the respective characteristic areas.

The control signal generating section 520 generates control signals fordriving the respective controllable elements based on the variouscontrol parameters calculated in the calculating section 510, andincludes a dynamic range control signal generator 521, a sensor exposuretime control signal generator 522, a shutter control signal generator523, a zoom/focus control signal generator 524, and an aperture controlsignal generator 525.

The dynamic range control signal generator 521 generates a drive signalfor the image sensor 30 for controlling the output level point or theinflection point at which the photoelectric conversion characteristic isswitched from the linear characteristic area to the logarithmiccharacteristic area based on the photoelectric conversion characteristicsetting value of the image sensor 30 which has been calculated in thedynamic range control parameter calculator 512, and sends the drivesignal to the timing generating circuit 31. The timing generatingcircuit 31 generates a timing signal for controlling the dynamic rangeof the image sensor 30 in response to the inputted drive signal, anddrives the image sensor 30. Specifically, as will be described later,the photoelectric conversion characteristic of the image sensor 30 has aproperty that the inflection point is changed by controlling a signalφVPS to the image sensor 30, namely, the intensity of the voltage VPH orthe duration of the time ΔT of the signal φVPS. In view of this, thedynamic range control signal generator 521 controls the dynamic range ofthe image sensor 30 in accordance with the subject luminance bycontrolling the drive signal inputted to the timing generating circuit31 for controlling the signal φVPS.

The sensor exposure time control signal generator 522 generates acontrol signal for controlling the exposure time, namely, theintegration time of the image sensor 30 by controlling operations of theelectronic circuitry, not by mechanical manipulation of the diaphragm22, the shutter 23, or a like device. The sensor exposure time controlsignal generator 522 generates a drive signal for the image sensor 30,specifically, a signal for controlling a time ΔS, with which the signalφVPS can attain a middle potential M, to secure a predetermined exposuretime, based on the optimal exposure amount calculated by the exposureamount control parameter calculator 511, and sends the drive signal tothe timing generating circuit 31. The timing generating circuit 31generates a timing signal for controlling the exposure time of the imagesensor 30 in response to the inputted drive signal, and drives the imagesensor 30.

Similarly, the shutter control signal generator 523 generates a drivesignal for setting the shutter speed of the shutter 23 in accordancewith the exposure time based on the optimal exposure amount calculatedby the exposure amount control parameter calculator 511. The zoom/focuscontrol signal generator 524 generates a control signal for driving thelens group 21 based on the optimal focal length calculated by the AFcontrol parameter calculator 513. Further, the aperture control signalgenerator 525 generates a control signal for setting the aperture areaof the diaphragm 22 based on the optimal exposure amount calculated bythe exposure amount control parameter calculator 511. The controlsignals generated in the shutter control signal generator 523, thezoom/focus control signal generator 524, and the aperture control signalgenerator 525 are sent to the corresponding elements of the drivingsection 60, respectively.

The input/output section 530 is connected to the memory card I/F 412 andto the LCD display I/F 413, and executes input/output operations such asrecording an image signal representing a captured image in the memorycard 107, displaying the captured image on the LCD section 106, orreading out the image signal from the memory card 107 after implementinga predetermined image processing with respect to the captured image inresponse to a command signal or the like sent from the operating section100.

Referring back to FIG. 2, the driving section 60 drives mechanicaldriving elements equipped in the digital camera 1, based on the controlsignals generated in the control signal generating section 520, andincludes a shutter driver 61, a zoom/focus driver 62, and an aperturedriver 63.

The shutter driver 61 drivingly opens and closes the shutter 23 to openthe shutter 23 for a predetermined time in response to a control signalsent from the shutter control signal generator 523. The zoom/focusdriver 62 drives a motor for operating the zoom lens block or a focuslens block of the lens group 21 in response to the control signal sentfrom the zoom/focus control signal generator 524 to move the lens blockto a focal point. The aperture driver 63 drives the diaphragm 22 inresponse to a control signal sent from the aperture control signalgenerator 525 to set the aperture amount of the diaphragm 22 to apredetermined value.

(Description on Overall Flow of Operation)

An overall flow is described on the operation of the digital camera 1having the above construction. FIG. 5 is a flowchart showing an exampleof the overall operation of the digital camera 1. As shown in FIG. 5,the operation of the digital camera 1 roughly comprises an evaluationvalue detecting step (step S1) of detecting evaluation values such as AEevaluation values, AF evaluation values, and WB evaluation values, acontrol parameter calculating step (step S2) of calculating variouscontrol parameters based on the evaluation values, and a controlparameter setting step (step S3) of setting the control parameters fordriving the respective elements of the digital camera 1, so that thedigital camera 1 is brought to a photographable state corresponding tothe control parameters.

In this embodiment, the aforementioned operation flow has features thatthe subject luminance for exposure setting is determined based on the AEevaluation values, and that AE control is performed in such a mannerthat the output of the image sensor 30 corresponding to the subjectluminance for exposure setting is obtained from the linearcharacteristic area of the image sensor 30 in calculating a controlparameter for AE control in step S2 based on the AE evaluation valuesdetected in step S1. In the following, the respective steps S1 throughS3 are described one by one by highlighting the features when needarises to do so.

In steps S1 through S3, the following processings are implemented.First, in the evaluation value detecting step S1, information relatingto evaluation values based on which various controls are implemented isacquired, and evaluation values are calculated based on the evaluationvalue information. In case of AE control, the luminance level of thesubject whose image is to be captured is measured or detected, and AEevaluation values are calculated based on the measured luminance level.The luminance level and the luminance range are detected as followssince it is rational to detect the luminance level and the luminancerange based on the subject image that has been actually captured by theimage sensor 30, and the image sensor 30 can shoot both still images andmoving images. In view of this, there are proposed two sub-steps S1-1and S1-2 as a step of acquiring luminance information:

(Step S1-1) Detection Based on Still Image:

A still image captured by the image sensor 30 before actual shooting isused as the image for detecting the evaluation values to measure theluminance level and the luminance range; and

(Step S1-2) Detection Based on Moving Image:

A moving image captured by the image sensor 30 before actual shooting isused as the image for detecting the evaluation values to measure theluminance level and the luminance range. Thereafter, the followingsub-step S1-3 is carried out.

(Step S1-3) Calculation of Evaluation Value:

Various evaluation values including AE evaluation values are calculatedby the evaluation value detector 405 based on the acquired luminanceinformation.

Next, in step S2, various parameters are calculated based on theevaluation values. Since the exposure amount or the dynamic range is aparameter for AE control, these control parameters are calculated basedon the AE evaluation values. Specifically, as the step S2, there areproposed two sub-steps S2-1 and S2-2 of calculating parameters:

(Step S2-1) Calculation of Exposure Amount Control Parameter:

An exposure amount control parameter is calculated by the maincontroller 50 based on the AE evaluation values; and

(Step S2-2) Calculation of Dynamic Range Control Parameter:

A dynamic range control parameter is calculated by the main controller50 based on the AE evaluation values.

Lastly, in step S3, the control parameters for driving the respectiveelements of the digital camera 1 are set. In case of AE control, thecontrol parameter setting is conducted based on sub-step S2-1 or S2-2.Accordingly, there are proposed two sub-steps S3-1 and S3-2 of settingparameters:

(Step S3-1) Setting of Exposure Amount Control Parameter:

The parameters for the memory unit 515, the control signal generatingsection 520 and the like are set based on the calculated exposure amountcontrol parameter to operate the timing generating circuit 31 and thedriving section 60; and

(Step S3-2) Setting of Dynamic Range Control Parameter:

The parameters for the memory unit 515, the control signal generatingsection 520 and the like are set based on the calculated dynamic rangecontrol parameter to operate the timing generating circuit 31.

(Basic Characteristics of Image Sensor to be Used in Embodiment)

In the following the steps are described one by one in detail. Firstly,an example of basic characteristics of the image sensor 30 to be used inthe embodiment is described in detail, in light of a fact that theembodiment is described based on the premise that the image sensor 30has a linear characteristic area where the electrical signal is linearlyconverted according to the amount of incident light, and a logarithmiccharacteristic area where the electrical signal is logarithmicallyconverted according to the amount of incident light.

FIG. 7 is an illustration schematically showing a two-dimensional MOStype solid-state image sensing device, as an example of the image sensor30. In FIG. 7, elements G11 through Gmn are pixels arrayed in a matrix.A vertical scanning circuit 301 and a horizontal scanning circuit 302are arranged in proximity to an outer perimeter of the pixel regioncomprised of the pixels G11 through Gmn. The vertical scanning circuit301 successively scans signal lines 304-1, 304-2, . . . , and 304-narrayed in row direction. Hereinafter, the group of the signal lines304-1, 304-2, . . . , and 304-n is called as “row-direction signal lineunit 304”. The horizontal scanning circuit 302 successively reads outphotoelectric conversion signals which have been outputted from therespective pixels to output signal lines 306-1, 306-2, . . . , and 306-mpixel by pixel in horizontal direction. Hereinafter, the group of theoutput signal lines 306-1, 306-2, . . . , and 306-m is called as “outputsignal line unit 306”. A power is supplied to the respective pixels by apower source line 305. Although other lines such as a clock line areconnected to the respective pixels, in addition to the row-directionsignal line unit 304, the power source line 305, and the output signalline unit 306, illustration of these other lines is omitted in FIG. 7.

Constant current sources 307-1, 307-2, . . . , and 307-m (hereinafter,called as “constant current source unit 307” as a group) are arranged incorrespondence to the output signal lines 306-1, 306-2, . . . , and306-m, respectively. Each of the constant current sources 307-1, 307-2,. . . , and 307-m, and a transistor T5, which will be described later,constitute an amplifying circuit. A resistor or a transistor such as aMOS transistor may constitute an amplifying circuit, in place of theconstant current source. Image data outputted from the respective pixelsin an image sensing operation by way of the output signal line unit 306,and correction data to be outputted in resetting are successivelyoutputted to sample hold circuits 308-1, 308-2, . . . , and 308-m(hereinafter, called as “selection circuit 308” as a group). The imagedata and the correction data are outputted row by row to the selectioncircuit 308 for sample-holding. The sample-held image data and thecorrection data are outputted to a correction circuit 309 column bycolumn. The correction circuit 309 corrects the image data based on thecorrection data, so that a noise component arising from sensitivityvariation is removed. After sensitivity variation of the respectivepixels has been corrected, the correction circuit 309 serially outputsthe image data pixel by pixel.

FIG. 8 is an illustration showing an example of a circuit configurationof each of the pixels G11 through Gmn. As shown in FIG. 8, each pixel iscomprised of a photodiode PD, transistors T1 through T6, each of whichis a metal oxide semiconductor field effect transistor (MOSFET), and acapacitor C for integration. A p-channel MOSFET is adopted as thetransistors T1 through T6. The symbols φVD, φV, φVPS, φRST, φS, and RSBrepresent signals or voltages to the respective transistors T1 throughT6 and to the capacitor C, and GND represents the ground.

The photodiode PD is a light sensing section or a photoelectricconversion section, and outputs an electrical signal, namely, aphotocurrent IPD commensurate with the amount of incident light from asubject. The transistor T5 and each of the constant current sourcesshown in FIG. 7 constitute an amplifying circuit, which is a sourcefollower circuit or a source follower amplifier to amplify a voltageVOUT, which will be described later, namely, to conduct currentamplification. The transistor T6 is a transistor for reading out asignal, and serves as a switch which is turned on and off in response toa voltage applied to a gate thereof. Specifically, a source of thetransistor T6 is connected to the output signal line unit 306 shown inFIG. 7, and the electric current which has been amplified by thetransistor T5 is drawn to the output signal line unit 306, as an outputcurrent, when the transistor T6 is turned on.

The transistor T2 generates at a gate thereof a voltage obtained bylinear conversion or log conversion of the photocurrent IPD. The MOSFETis designed in such a manner that a minute current called a subthresholdcurrent flows when the gate voltage is not larger than a thresholdvalue. The transistor T2 conducts the linear conversion or the logconversion by utilizing the subthreshold characteristic.

Specifically, if the subject luminance is low, or the subject is dark,namely, if the amount of light to be incident onto the photodiode PD issmall, the gate potential of the transistor T2 is higher than the sourcepotential thereof. Accordingly, the transistor T2 is in a so-called“cutoff state”, and a subthreshold current does not flow in thetransistor T2, namely, the transistor T2 is not operated in thesubthreshold region. As a result, the photocurrent generated in thephotodiode PD flows to the parasitic capacitance of the photodiode PD tothereby accumulate electric charge therein, and a voltage correspondingto the accumulated electric charge is generated. At this time, since thetransistor T1 is kept in an ON state, a voltage corresponding to theelectric charge accumulated in the parasitic capacitance of thephotodiode PD is generated at the gates of the transistors T2 and T3 asa voltage VG. Because of generation of the voltage VG, an electriccurrent flows in the transistor T3, and electric charge proportional tothe voltage VG is accumulated in the capacitor C. The transistor T3 andthe capacitor C constitute an integration circuit. As a result, avoltage which is linearly proportional to the integration value of thephotocurrent IPD is obtained at the connection node a of the transistorT3 and the capacitor C, namely at the output VOUT. At this time, thetransistor T4 is in an OFF state. In response to turning on of thetransistor T6, the electric charge accumulated in the capacitor C isdrawn to the output signal line unit 306 as an output current via thetransistor T5. The output current is a value obtained by linearconversion of the integration value of the photocurrent IPD. This is howthe image sensor 30 is operated in the linear characteristic area.

On the other hand, if the subject luminance is high or the subject isbright, namely, if the amount of light to be incident onto thephotodiode PD is large, the gate potential of the transistor T2 is notlarger than the source potential thereof, and a subthreshold currentflows in the transistor T2, namely, the transistor T2 is operated in thesubthreshold region. As a result, a voltage VG obtained bynatural-logarithmic conversion of the photocurrent IPD is generated atthe gates of the transistors T2 and T3. Because of generation of thevoltage VG, an electric current flows in the transistor T3, and electriccharge equivalent to the value obtained by natural-logarithmicconversion of the integration value of the photocurrent IPD isaccumulated in the capacitor C. As a result, a voltage which isproportional to the value obtained by natural-logarithmic conversion ofthe integration value of the photocurrent IPD is generated at theconnection node a or at the output VOUT of the capacitor C and thetransistor T3. At this time, the transistor T4 is in an OFF state. Then,in response to turning on of the transistor T6, the electric chargeaccumulated in the capacitor C is drawn to the output signal line unit306 as an output current via the transistor T5. The output current isthe value obtained by natural-logarithmic conversion of the integrationvalue of the photocurrent IPD. This is how the image sensor 30 isoperated in the logarithmic characteristic area. As mentioned above, avoltage linearly or natural-logarithmically proportional to the amountof incident light, namely, the subject luminance is outputted withrespect to each of the pixels.

The transistor T1 is a switch to be used in extracting noise data or anoise signal arising from a variation in production of transistors T2 atthe time of resetting. The transistor T1 is kept in an ON state exceptfor a reset time, and is designed to flow the photocurrent IPD betweenthe drain of the transistor T2 and the photodiode PD. At the time ofresetting, the transistor T2 is brought to an OFF state to shut off flowof the photocurrent IPD through the photodiode PD. Thereby, merely thenoise data resulting from the production variation is extracted. Theextracted variation component or a noise signal is subtracted from avideo signal, which will be described later.

The transistor T4 is a transistor for resetting the capacitor C, andserves as a switch which is turned on and off in response to a voltageapplied to the gate of the transistor T4. In response to turning on ofthe transistor T4, a reset voltage or a voltage of the signal RSB isapplied to the transistor T4 to thereby return the capacitor C to aninitial state before accumulation of electric charge, namely, to a statebefore start of integration.

FIG. 9 is an in illustration showing a timing chart on an image sensingoperation of the image sensor 30, namely, a pixel. In this embodiment,in light of polarities of the p-channel MOSFET, the transistor is turnedoff when the respective signals are set high (Hi), and turned on whenthe respective signals are set low (Low). First, when the signal φV isset Low at the timing indicated by the arrow 311, the transistor T6 isturned on to read out a video signal. Specifically, the electric chargeaccumulated in the capacitor C is drawn to the output signal line unit306 as an output current or a video signal. Then, when the signal φS isset Hi at the timing indicated by the arrow 312, the transistor T1 isturned off to disconnect the photodiode PD. Subsequently, when thesignal φVPS is set Hi at the timing indicated by the arrow 313, thetransistor T2 is reset. Further, concurrently with the resetting of thetransistor T2, the signal φRST is set Low at the timing indicated by thearrow 314, and the transistor T4 is turned on. Thereby, a reset voltageis applied to the capacitor C at the connection node a by the signalRSB, namely, the potential at the connection node a becomes thepotential VRSB of the signal RSB to thereby reset charge accumulation ofthe capacitor C. Thus, after resetting of the transistor T2 and thecapacitor C, when the signal φV is set Low at the timing indicated bythe arrow 315, the transistor T6 is turned on to thereby draw the noisesignal to the output signal line unit 306.

Next, when the signal φS is set Low at the timing indicated by the arrow316, the transistor T1 is turned on to thereby release disconnection ofthe photodiode PD. Then, when the signal φVPS is set to the middlepotential M at the timing indicated by the arrow 318, the parasiticcapacitance of the photodiode PD is reset for reducing signal residue.Then, when the signal φRST is set Low at the timing indicated by thearrow 317 to make a voltage for integration start in a next frameconstant, the transistor T4 is turned on to reset charge accumulation ofthe capacitor C.

Thereafter, when the signal φVPS is set Low at the timing indicated bythe arrow 319, resetting of the parasitic capacitance of the photodiodePD is terminated. Concurrently, the signal φRST is set Hi to terminatethe resetting operation of the capacitor C. The integration time of thecapacitor C is started at the timing t1, and continued for a duration upto the timing of the signal φV indicated by the arrow 311, namely, up tothe timing t2 at which readout of a video signal in the next frame isstarted. The time duration from the timing t1 to the timing t2corresponds to an integration time of the capacitor C, namely, anexposure time for an image sensing. The exposure time is controlled bycontrolling the time ΔS with which the signal φVPS can achieve themiddle potential M. The time duration ΔS is controlled by the sensorexposure time control signal generator 522 by way of the timinggenerating circuit 31.

The signal φVD is used to control the potential in such a manner thatthe potential lies in an operation range of the amplifying circuit,namely, the source follower amplifier, or to perform offset adjustmentwith respect to a video signal or a noise signal. Vh, Vm, and Vl of thesignal φVD respectively represent high potential, middle potential, andlow potential of the signal φVD.

As mentioned above, the image sensor 30 is capable of acquiring anoutput signal obtained by linear conversion or log conversion accordingto the subject luminance, and has a photoelectric conversioncharacteristic 320 as shown in FIG. 10. As shown in FIG. 10, thephotoelectric conversion characteristic 320 is divided into a linearcharacteristic area and a logarithmic characteristic area, with aninflection point 321 serving as a boundary. The inflection point 321 isa switching point from the linear characteristic area to the logarithmiccharacteristic area, and the output value of the image sensor 30 at theinflection point 321 is represented by Vth. Generally, in the linearcharacteristic area, high gradation performance can be secured withrespect to the entirety of an image, namely, high contrast is obtained,although sensing of a subject in a wide luminance range is impossible,namely, the dynamic range is narrow. Accordingly, an image with highgradation performance and high quality can be obtained even from a darksubject, e.g., in a condition that a subject is captured in a cloudyweather or in a shadow. On the other hand, in the logarithmiccharacteristic area, sensing of a subject image in a wide luminancerange is possible, namely, the dynamic range is wide, although thegradation is poor at a high luminance. Accordingly, a high-quality imagewith a large depth of field including a dark area can be obtained evenif a subject is bright, e.g., in a condition that a subject isilluminated with direct sunlight, or direct sunlight is right behind thesubject.

The photoelectric conversion characteristic 320, namely, the inflectionpoint 321 can be changed or moved by changing a voltage difference ofthe signal φVPS inputted to the source of the transistor T2 between Hiand Low. Specifically, assuming that the Hi voltage of the signal φVPSis VPH, and the Low voltage thereof is VPL, then, as shown in FIG. 11,the photoelectric conversion characteristic 320, namely, the inflectionpoint 321 can be desirably shifted to a photoelectric conversioncharacteristic 322, namely, to an inflection point 324 or to aphotoelectric conversion characteristic 323, namely, to an inflectionpoint 325 by changing the voltage difference ΔVPS (=VPH−VPL) (see FIG.9). Thus, by changing the photoelectric conversion characteristic, theratio of the linear characteristic area to the logarithmiccharacteristic area can be changed, whereby the photoelectric conversioncharacteristic having a relatively large ratio of the linearcharacteristic area, as exemplified by the photoelectric conversioncharacteristic 322, or the photoelectric conversion characteristichaving a relatively large ratio of the logarithmic characteristic area,as exemplified by the photoelectric conversion characteristic 323 can beobtained. The photoelectric conversion characteristic may be changed insuch a manner that the linear characteristic area or the logarithmiccharacteristic area occupies the entirety of the photoelectricconversion characteristic.

In this embodiment, the photoelectric conversion characteristic of theimage sensor 30 is changed by changing the voltage difference ΔVPS,namely, by changing the voltage VPH. In FIG. 11, as the voltage VPH isincreased, namely, the voltage difference ΔVPS is increased, the ratioof the linear characteristic area is increased, and consequently, thephotoelectric conversion characteristic of the image sensor 30approaches the photoelectric conversion characteristic 322. On the otherhand, as the voltage VPH is decreased, namely, the voltage differenceΔVPS is decreased, the ratio of the logarithmic characteristic area isincreased, and consequently, the photoelectric conversion characteristicof the image sensor 30 approaches the photoelectric conversioncharacteristic 323. The voltage VPH is controlled by the dynamic rangecontrol signal generator 521 by way of the timing generating circuit 31.

Alternatively, the time ΔT during which the signal φVPS corresponding tothe voltage VPH is applied may be changed to change the photoelectricconversion characteristic as mentioned above. In such an alteredarrangement, as the time ΔT is increased, the photoelectric conversioncharacteristic is changed in such a manner that the ratio of the linearcharacteristic area is increased, and conversely, as the time ΔT isdecreased, the photoelectric conversion characteristic is changed insuch a manner that the ratio of the logarithmic characteristic area isincreased. In FIG. 11, the state that the time ΔT is relatively longcorresponds to the photoelectric conversion characteristic 322, and thestate that the time ΔT is relatively short corresponds to thephotoelectric conversion characteristic 323.

(Evaluation Value Detecting Step S1)

Next, an exemplified process for acquiring evaluation values such as AEevaluation values by the evaluation value detector 405 of the signalprocessing section 40 is described.

(Step S1-1) Example of Detecting Evaluation Value Based on Still Image:

FIG. 12 is a flowchart illustrating an operation example of the digitalcamera 1 in the case where evaluation values such as AE evaluationvalues of a subject are detected based on a still image actuallycaptured by the image sensor 30. Specifically, FIG. 12 shows a flow ofpreliminary shooting of capturing a still image for acquiring AEevaluation values, and calculating the AE evaluation values based on theimage captured by the preliminary shooting before actual shooting ofcapturing a still image by the digital camera 1 according to theembodiment of the invention. The evaluation value detecting process is aprocess suitable for an image sensing apparatus such as a digital singlelens reflex camera constructed such that a subject light image isincident onto an optical viewfinder without incident onto the imagesensor 30 in preliminary shooting.

Referring to FIG. 12, first, it is confirmed whether shooting start hasbeen designated in a state that the power supply switch 101 of thedigital camera 1 is pressed, and the power of the digital camera 1 isturned on (Step S111). In response to manipulation of the release switch102 e.g., halfway down pressing (YES in Step S111), an operation ofpreliminary shooting is initiated (Step S112).

In Step S112, dynamic range control for the preliminary shooting isconducted in conducting the preliminary shooting to calculate the AEevaluation values. The dynamic range control is conducted, so that theimage sensor 30 has a possible maximum dynamic range to sense thesubject luminance in a wide range. Specifically, since the preliminaryshooting is conducted only one time before actual shooting in thedigital single lens reflex camera or a like apparatus, the digitalcamera 1 is constructed in such a manner that a wide dynamic range isset to securely detect the subject luminance in whatsoever condition thesubject may be.

In view of the above, the operation state of the image sensor 30 iscontrolled, so that the image sensor 30 implements a log conversionoutput operation in the entirety of the photoelectric conversioncharacteristic. Specifically, in response to halfway down pressing ofthe release switch 102, the main controller 50 outputs commands to therelevant elements to shift the digital camera 1 to the preliminaryshooting mode. Upon receiving the command signal from the maincontroller 50, for example, the dynamic range control signal generator521 generates a signal for varying the voltage difference ΔVPS of thesignal φVPS inputted to the source of the transistor T2 as shown in FIG.8. In this case, the signal is adapted for reducing the voltagedifference ΔVPS (see FIG. 9). Thereby, the image sensor 30 is controlledin such a manner that the ratio of the logarithmic characteristic areais increased. Preferably, a logarithmic characteristic area occupies theentirety of the photoelectric conversion characteristic in the aspect ofsecuring a wide dynamic range. However, it is possible to leave a linearcharacteristic area in the photoelectric conversion characteristic inplace of allowing a logarithmic characteristic area to occupy theentirety of the photoelectric conversion characteristic.

Next, exposure control for preliminary shooting is conducted to executethe preliminary shooting (Step S113). For instance, the sensor exposuretime control signal generator 522 generates a drive signal for settingthe duration of the time ΔS, with which the signal φVPS can achieve amiddle potential M, to a predetermined exposure time, and sends thedrive signal to the timing generating circuit 31 to conduct exposurecontrol or exposure amount control for preliminary shooting. Exposurecontrol may be performed by causing the shutter driver 61 to control theshutter speed of the shutter 23 based on a control signal generated inthe shutter control signal generator 523, and by causing the aperturedriver 63 to control the diaphragm 22 based on a control signalgenerated in the aperture control signal generator 525. After theexposure control, the preliminary shooting of a still image isconducted. Then, the evaluation value detector 405 calculates AEevaluation values based on the image captured by the preliminaryshooting (Step S114). The AE evaluation value calculating step will bedescribed later in detail. After the calculation of the AE evaluationvalues, the preliminary shooting is terminated (Step S115). After theexposure control based on the AE evaluation values, actual shooting isstarted (Step S116). The above is what is to be implemented in acquiringthe AE evaluation values. Similar steps are implemented to acquire theAF evaluation values and the WB evaluation values.

(Step S1-2) Example of Detecting Evaluation Values Based on MovingImages:

FIG. 13 is a flowchart illustrating an operation example of the digitalcamera 1 in detecting evaluation values such as AE evaluation values ofa subject based on a moving image that is continuously captured by theimage sensor 30. Specifically, FIG. 13 shows a flow of calculating theAE evaluation values with use of all the frame images captured by theimage sensor 30 in the case where the digital camera 1 is in a shootingstandby state, or in the moving image shooting mode, or in the casewhere the image sensing apparatus according to the embodiment of theinvention is applied to a digital movie camera.

Referring to FIG. 13, it is confirmed whether shooting start has beendesignated (Step S121). For instance, if the mode setting switch 105 ispressed to shift the digital camera 1 to the moving image shooting mode,and shooting start is confirmed (YES in Step S121), shooting of a movingimage is initiated (Step S122). Respective control values on the imagesensing dynamic range, the exposure time, and the aperture value at thetime of shooting the moving image are set to initial values.

Subsequently, the evaluation value detector 405 calculates the AEevaluation values based on the image captured in Step S122 (Step S123).Then, based on the detected AE evaluation values, the dynamic rangecontrol signal generator 521 changes the setting of the signal φVPS tocontrol the dynamic range, and the shutter 23 and the diaphragm 22 arecontrolled by control signals generated by the shutter drive signalgenerator 523 and the aperture control signal generator 525, whereby aspecified AE control is carried out (Step S124)

Then, it is confirmed whether the shooting has been terminated (StepS125). If there is no command signal indicative of shooting termination(NO in Step S125), the routine returns to Step S123 to repeat the AEevaluation value calculation, and the AE control in Step S124.Specifically, in the moving image shooting, the steps of utilizing allthe captured images as evaluation images for detecting the AE evaluationvalues, and conducting the AE control for next shooting based on thedetected AE evaluation values are cyclically repeated. Alternatively,part of the captured images, e.g., one frame image per several frameimages corresponding to the captured images, may be used as anevaluation image, in place of using all the captured images asevaluation images, and AE evaluation values may be acquired based on theevaluation image.

(Step S1-3) Calculation of Evaluation Values:

Next, the evaluation value calculating step in the above flow, namely,Steps S113 and S123, are described in detail. FIG. 14 is a block diagramof the evaluation value detector 405. The evaluation value detector 405includes a multi-pattern metering section 4051, a histogram calculatingsection 4052, and a saturation judging section 4055.

The multi-pattern metering section 4051 conducts metering of a subjectaccording to a multi-pattern metering system. Specifically, themulti-pattern metering section 4051 divides an image captured by theimage sensor 30 into areas and blocks of a predetermined number, anddetects the luminance of the captured image in the respective areas andblocks, based on image signals or image data.

FIG. 15 is an illustration showing a state as to how the image sensingarea to be metered is divided by multi-pattern metering. Denoted by thereference numeral 330 is an image sensing area obtained by an imagesensing operation by the image sensor 30. A subject image is captured orsensed within the image sensing area 330. The image sensing area 330carries a multitude of pixel information concerning pixels constitutingthe image sensor 30, namely, luminance information concerning a subjectimage. The image sensing area 330 is divided into a central area, whichis a central part of the image sensing area 330, and a peripheral area,which is a peripheral part around the central part. Further, the centralarea and the peripheral area are each divided into detection blocks of apredetermined number. For instance, the central area is divided intothirty-six detection blocks comprised of A, B, C, . . . Z, AA, . . . ,and AJ, namely, detection blocks A through AJ, and the peripheral areais divided into sixteen detection blocks comprised of first throughsixteenth detection blocks. In this embodiment, a subject image capturedin the central area is called as “main subject image”, a subject imagecaptured in the peripheral area is called as “peripheral subject image”,the central area is called as “main subject image area 331”, and theperipheral area is called as “peripheral subject image area 332”. Thearea defined by the detection blocks O, P, U, and V in the central partof the main subject image area 331 is called as “AF detecting area 333”where AF evaluation values are detected for focusing control. Further,the luminance of the captured image in the main subject image area 331is called as “main subject luminance”, and the luminance of the capturedimage in the peripheral subject image area 332 is called as “peripheralsubject luminance”.

The histogram calculating section 4052 calculates a histogram, namely,distribution of a main subject luminance with respect to each of thedetection blocks A through AJ, and calculates a histogram of the mainsubject luminance in the entirety of the main subject image area 331,namely, a main subject entire luminance histogram, as shown in FIG. 16A,with use of the main subject luminance histograms with respect to thedetection blocks A through AJ. Further, the histogram calculatingsection 4052 calculates a histogram of a peripheral subject luminancewith respect to each of the first through sixteenth detection blocks,and calculates a histogram of the peripheral subject luminance in theentirety of the peripheral subject image area 332, namely, a peripheralsubject entire luminance histogram, as shown in FIG. 16B, with use ofthe peripheral subject luminance histograms with respect to the firstthrough sixteenth detection blocks.

Further, the histogram calculating section 4052 calculates the luminancerange of the entirety of the main subject image and the luminance rangeof the entirety of the peripheral subject image with use of the mainsubject entire luminance histogram and the peripheral subject entireluminance histogram, respectively. In calculating, a so-called “Gaussianpruning” is applied using a specified threshold value. For the mainsubject, the luminance data is cut back at a threshold value D1 as shownin FIG. 16A, and a range defined by a minimum L1 and a maximum value L8of luminances having frequencies equal to or above D1 is set as a mainsubject entire luminance range. Similarly, for the peripheral subject,the luminance data is cut back at a threshold value D2 as shown in FIG.16B, and a range defined by a minimum value L12 and a maximum value L19of luminances having frequencies equal to or above D2 is set as aperipheral subject entire luminance range. This Gaussian pruning usingthe threshold values is applied for reducing errors caused by noise orthe like. Although the luminances of the respective luminance histogramsshown in FIGS. 16A and 16B are identified by L1 to L19 here for the sakeof convenience, they are actually expressed in 256 stages or gradationsand can be identified by L1 to L256, for example, in the case ofhandling image data of eight bits.

The histogram calculating section 4052 has an average luminancecalculator 4053 and a maximum/minimum luminance calculator 4054. Theaverage luminance calculator 4053 calculates an average luminance of themain subject image with respect to each of the detection blocks Athrough AJ, and an average luminance of the peripheral subject imagewith respect to each of the first through sixteenth detection blocks.The average luminance is calculated with respect to each of the colorcomponents R, G, and B. In calculating the average luminances, ahistogram of the main subject luminance with respect to each of thedetection blocks A through AJ, and a histogram of the peripheral subjectluminance with respect to each of the first through sixteenth detectionblocks are calculated, and the Gaussian pruning with use of specifiedthreshold values is applied in a similar manner as calculating the mainsubject entire luminance range and the peripheral subject entireluminance range. The average luminance of the main subject image withrespect to each of the detection blocks A through AJ, and the averageluminance of the peripheral subject image with respect to each of thefirst through sixteenth detection blocks are obtained by averaging theluminances after the Gaussian pruning.

The maximum/minimum luminance calculator 4054 calculates amaximum/minimum luminance of the main subject image with respect to eachof the detection blocks A through AJ, and a maximum/minimum luminance ofthe peripheral subject image with respect to each of the first throughsixteenth detection blocks. Similarly to the above, the Gaussian pruningwith use of predetermined threshold values is applied with respect tothe main subject luminance histograms and the peripheral subjectluminance histograms which have been calculated with respect to thedetection blocks A through AJ, and the first through sixteenth detectionblocks, and the maximum/minimum luminance of the main subject image withrespect to each of the detection blocks A through AJ, and themaximum/minimum luminance of the peripheral subject image with respectto each of the first through sixteenth detection blocks are calculatedbased on the luminances or luminance ranges after the Gaussian pruning.

The histogram calculating section 4052 calculates a subject image entireluminance histogram in the entire image sensing area, namely, the imagesensing area 330, based on the main subject entire luminance histogramand the peripheral subject entire luminance histogram, so that thesubject image entire luminance histogram can be used in judging thesaturation by the saturation judging section 4055, which will bedescribed later. The saturation judging section 4055 judges whetheroutput of the image sensor 30 has been saturated at the time ofdetecting the AE evaluation values, or the AF evaluation values, or theWB evaluation values based on the subject image entire luminancehistogram calculated by the histogram calculating section 4052.

FIG. 17 is an illustration showing an example of the subject imageentire luminance histogram at the time of saturation. In FIG. 17, Pmaxrepresents an incident luminance or a saturation luminance of the imagesensor 30 when the image sensor 30 reaches the saturation output levelVmax, which is a physically maximum value in output level of the imagesensor 30, and Pmaxth represents an incident luminance or a luminancethreshold of the image sensor 30 relative to sensor output Vmaxth, whichis set as a threshold value for judging whether the output of the imagesensor 30 is saturated or not. Dth represents a frequency or a frequencythreshold which has been set as a threshold value for judging whetherthe output of the image sensor 30 is saturated or not.

The saturation judging section 4055 calculates the total frequency in ahatched area (hereinafter, called as “saturated area 342”) indicated bythe reference numeral 342 in FIG. 17, wherein the luminance is notsmaller than the luminance threshold Pmaxth and not smaller than thefrequency threshold Dth, namely, the total number of pixels in thesaturated area 342, which is called as “saturated pixel number”. Thesaturation judging section 4055 judges that the output level of theimage sensor 30 is saturated if the saturated pixel number is notsmaller than a predetermined value. On the other hand, the saturationjudging section 4055 judges that the output level of the image sensor 30is not saturated if the saturated pixel number is smaller than thepredetermined value. Alternatively, judgment as to whether the outputlevel of the image sensor 30 is saturated may be made based solely onthe frequency of the saturated luminance Pmax, namely, based on thepixel number having the saturated luminance Pmax.

As mentioned above, the evaluation value detector 405 performs themulti-pattern metering, and detects information relating to the averageluminances, the maximum/minimum luminances, the luminance histograms,the luminance ranges, or a like parameter, as the AE evaluation valuesor the AF evaluation values, or the WB evaluation values based on theluminance information or image data in each of the detection blocks ofthe main subject image area 331 and the peripheral subject image area332. The data concerning the evaluation values are outputted to therelevant parameter calculators of the calculating section 510. Forinstance, if the evaluation values are AE evaluation values, the AEevaluation values are outputted to the AE control parameter calculatingunit 5110. If the evaluation values are AF evaluation values, the AFevaluation values are outputted to the AF control parameter calculator513. If the evaluation values are WB evaluation values, the WBevaluation values are outputted to the WB control parameter calculator514. Upon receiving the respective evaluation values, the respectivecalculators calculate the control parameters based on the evaluationvalues.

(AE Control Parameter Calculating Step S2)

In this embodiment, a subject luminance for exposure setting isdetermined based on the detected AE evaluation values, and the AEcontrol parameter calculating unit 5110 (see FIG. 4) calculates an AEcontrol parameter, so that the output of the image sensor 30corresponding to the subject luminance for exposure setting is obtainedfrom the linear characteristic area of the image sensor 30. The subjectluminance for exposure setting can be arbitrarily determined. However,it is desirable to select the AE evaluation values such as the luminancerange and the average luminance of the main subject image, which areacquired from the main subject image, to capture the main subject imagewith good contrast.

In performing AE control in this embodiment, a method of controlling theexposure amount to the image sensor 30, and a method of controlling thedynamic range are proposed as a method for obtaining the output of theimage sensor 30 corresponding to the subject luminance from the linearcharacteristic area of the image sensor 30. FIGS. 18A through 19 aregraphs showing how the photoelectric conversion characteristic of theimage sensor 30 is changed in performing AE control. FIGS. 18A and 18Bshow change of the photoelectric conversion characteristic in performingexposure amount control, and FIG. 19 shows change of the photoelectricconversion characteristic in performing dynamic range control. In FIGS.18A through 19, the axis of abscissas represents an incident luminanceto the image sensor 30, and the axis of ordinate represents an outputlevel of the image sensor 30, wherein the axis of abscissas isrepresented in terms of log coordinates, namely, log values of incidentluminances to the image sensor 30. Throughout the specification and theclaims, the incident luminance to the image sensor 30 means a subjectluminance that has been incident onto the image sensor 30, andhereinafter, simply called as “luminance”.

The exposure amount control and the dynamic range control are performedbased on the following controls (A) and (B), respectively.

(A) Exposure amount control based on control of the exposure time of theshutter 23 and/or the image sensor 30, namely, the opening time of theshutter 23 and/or the integration time of the image sensor 30, and/orthe aperture area of the diaphragm 22.

(B) Dynamic range control based on control of the photoelectricconversion characteristic of the image sensor 30, specifically, controlof the position of the inflection point of the photoelectric conversioncharacteristic (see FIG. 19).

(Step S2-1) Calculation of Exposure Amount Control Parameter:

First, exposure amount control (A) is described referring to FIGS. 18Aand 18B. FIG. 18A shows a case that the subject luminance for exposuresetting based on the AE evaluation values is located in the logarithmiccharacteristic area, and the exposure amount is controlled to conduct animage sensing operation in the linear characteristic area. FIG. 18Bshows a case that the subject luminance for exposure setting based onthe AE evaluation values is located in a relatively low output levelarea of the linear characteristic area, and the exposure amount iscontrolled to conduct an image sensing operation in a relatively highoutput level area of the linear characteristic area.

In FIG. 18A, a photoelectric conversion characteristic 601 is aphotoelectric conversion characteristic of the image sensor 30 at thetime of acquiring the AE evaluation values stored in the photoelectricconversion characteristic information storage 516. The photoelectricconversion characteristic 601 is divided into a linear characteristicarea and a logarithmic characteristic area, with the inflection point603, namely, the sensor output Vth, serving as a boundary. The exposureamount control parameter calculator 511 calculates an exposure amountcontrol parameter, namely, an exposure amount setting value forobtaining an exposure amount, based on which the photoelectricconversion characteristic 601 is changed to a photoelectric conversioncharacteristic 602 capable of obtaining a predetermined sensor outputcorresponding to the subject luminance for exposure setting. Namely, theexposure amount control parameter calculator 511 calculates an exposuretime setting value for controlling the exposure time, and an aperturesetting value for controlling the aperture area of the diaphragm 22. Inother words, the exposure amount control parameter is calculated basedon the subject luminance for exposure setting based on the AE evaluationvalues, and the photoelectric conversion characteristic 601 stored inthe photoelectric conversion characteristic information storage 516.

Here is calculated the photoelectric conversion characteristic 602 insuch a manner that the sensor output at the point 605 corresponding to aspecified luminance Lt1 for exposure amount setting in the linearcharacteristic area of the photoelectric conversion characteristic 601becomes a sensor output Vtarget at the point 606 in the linearcharacteristic area. In other words, the new photoelectric conversioncharacteristic 602 is obtained, so that the sensor output, namely, thepoint 605 corresponding to the luminance Lt1 in the logarithmiccharacteristic area of the photoelectric conversion characteristic 601coincides with the point 606 in the linear characteristic area in termsof graphical expression by changing or shifting the photoelectricconversion characteristic 601 to the photoelectric conversioncharacteristic 602 passing the point 606 in a direction shown by thearrow 608. At this time, the inflection point 603 is shifted in parallelto the inflection point 604, and the sensor output Vth does not change.The sensor output Vtarget is a target output, and is set to apredetermined value. The sensor output Vtarget is stored in the exposureamount control parameter calculator 511 or a like device.

More specifically, the exposure time setting value or the aperturesetting value capable of decreasing the exposure amount is calculated insuch a manner that the sensor output corresponding to the luminance Lt1is decreased from Vt1 at the point 605 in the logarithmic characteristicarea of the photoelectric conversion characteristic 601 to Vtarget atthe point 606, namely, the sensor output is decreased relative to thesame magnitude of luminance. In other words, the exposure amount controlparameter calculator 511 calculates the exposure time setting value orthe aperture setting value capable of decreasing the exposure amount insuch a manner that the luminance corresponding to the target outputVtarget is changed from Lt2 at the point 607 to Lt1, namely, theluminance for obtaining the target output Vtarget is not smaller thanthe predetermined value Lt1 (>Lt2). At the time of decreasing theexposure amount, the opening time of the shutter 23 or the integrationtime of the image sensor 30 is reduced, and the aperture area of thediaphragm 22 is reduced based on the exposure time setting value or theaperture setting value.

In shifting the photoelectric conversion characteristic from thephotoelectric conversion characteristic 601 to the photoelectricconversion characteristic 602, the luminance corresponding to Vmax isincreased from Lm2 to Lm1, and consequently, the dynamic range isincreased. Vmax is a maximal output value of the image sensor 30,namely, a saturated output level of the image sensor 30. The saturatedoutput level Vmax may be an arbitrarily set value, e.g., a valueslightly lower than the actual maximum value.

Next, in FIG. 18B, a photoelectric conversion characteristic 611 isphotoelectric conversion characteristic of the image sensor 30 at thetime of acquiring the AE evaluation values stored in the photoelectricconversion characteristic information storage 516. Similarly to thephotoelectric conversion characteristic 601, the photoelectricconversion characteristic 611 is divided into a linear characteristicarea and a logarithmic characteristic area, with the inflection point613, namely, the sensor output Vth, serving as a boundary.

Referring to FIG. 18B, the subject luminance Lt1 for exposure setting islocated in a relatively low output level area at the point 615 in thelinear characteristic area of the photoelectric conversioncharacteristic 611. Accordingly, the exposure control parametercalculator 511 calculates a photoelectric conversion characteristic 612in such a manner that the sensor output at the point 615 correspondingto the luminance Lt1 in the linear characteristic area of thephotoelectric conversion characteristic 611 becomes the target outputVtarget at the point 616. In other words, the new photoelectricconversion characteristic 612 is obtained in such a manner that thesensor output, namely, the point 615 corresponding to the luminance Lt1in the relatively low output level area of the linear characteristicarea of the photoelectric conversion characteristic 611 coincides withthe point 616 in a relatively high output level area of the linearcharacteristic area in terms of graphical expression by changing orshifting the photoelectric conversion characteristic from thephotoelectric conversion characteristic 611 to the photoelectricconversion characteristic 612 passing the point 616 in a direction shownby the arrow 618. At this time, the inflection point 613 is shifted inparallel to the inflection point 614, and the sensor output Vth does notchange.

More specifically, the exposure time setting value or the aperturesetting value capable of increasing the exposure amount is calculated insuch a manner that the sensor output corresponding to the luminance Lt1is increased from the sensor output at the point 615 to the targetoutput Vtarget at the point 616, namely, the sensor output is increasedrelative to the same magnitude of luminance. In other words, theexposure amount control parameter calculator 511 calculates the exposuretime setting value or the aperture setting value capable of increasingthe exposure amount in such a manner that the luminance Lt2 at the point617 corresponding to the target output Vtarget is shifted to Lt1,namely, the luminance for obtaining the target output Vtarget is assmall as Lt1(<Lt2). At the time of increasing the exposure amount, theopening time of the shutter 23 or the integration time of the imagesensor 30 is increased, and the aperture ratio of the diaphragm 22 isincreased based on the exposure time setting value or the aperturesetting value. As the photoelectric conversion characteristic 611 isshifted to the photoelectric conversion characteristic 612, theluminance corresponding to the sensor output Vmax is changed or loweredfrom Lm2 to Lm1, and consequently, the dynamic range is reduced.

In the aforementioned exposure amount control parameter calculatingstep, the photoelectric conversion characteristic is not changed orshifted if the photoelectric conversion characteristic at the time ofobtaining the AE evaluation values has a feature that the target outputVtarget can be already obtained in relation to the luminance forexposure amount setting as described above. However, in such a case,even if the exposure time setting value and the aperture setting valuetake the same values as those when the AE evaluation values wereobtained last time, the exposure time setting value and the aperturesetting value may be calculated this time.

(Step S2-2) Calculation of Dynamic Range Control Parameter:

Next, the dynamic range control (B) is described referring to FIG. 19.FIG. 19 shows a case that the subject luminance for exposure settingbased on the AE evaluation values is located in the logarithmiccharacteristic area, and the dynamic range is controlled to conduct animage sensing operation in the linear characteristic area. Referring toFIG. 19, a photoelectric conversion characteristic 701 is aphotoelectric conversion characteristic of the image sensor 30 at thetime of acquiring the AE evaluation values stored in the photoelectricconversion characteristic information storage 516. The photoelectricconversion characteristic 701 is divided into a linear characteristicarea and a logarithmic characteristic area, with the inflection point703, namely, the sensor output Vth1, serving as a boundary.

The dynamic range control parameter, namely, the photoelectricconversion characteristic setting value is a control value for thephotoelectric conversion characteristic, based on which thephotoelectric conversion characteristic 701 is changed to aphotoelectric conversion characteristic 702, so that a predeterminedsensor output corresponding to a predetermined luminance for setting thedynamic range, namely, the subject luminance for exposure setting isobtained. In other words, the subject luminance is obtained in terms ofsensor output in the linear characteristic area. The photoelectricconversion characteristic setting value is calculated by the dynamicrange control parameter calculator 512. Namely, the dynamic rangecontrol parameter is calculated based on the subject luminance forexposure setting based on the AE evaluation values, and thephotoelectric conversion characteristic 701 stored in the photoelectricconversion characteristic information storage 516.

The photoelectric conversion characteristic 702 is calculated to matchthe sensor output Vt1 at the point 705 in the logarithmic characteristicarea corresponding to the luminance Lt1, namely, the subject luminancefor exposure setting in the linear characteristic area of thephotoelectric conversion characteristic 701 with the target outputVtarget at the point 706 in the linear characteristic area. In otherwords, the dynamic range control parameter calculator 512 calculates aphotoelectric conversion characteristic setting value based on which thephotoelectric conversion characteristic 701 is changed or shifted to thenew photoelectric conversion characteristic 702 passing the point 706 ina direction shown by the arrow 707. Namely, the new photoelectricconversion characteristic 702 is calculated in such a manner that thesensor output, namely, the point 705 corresponding to the luminance Lt1in the logarithmic characteristic area of the photoelectric conversioncharacteristic 701 coincides with the point 706 in the linearcharacteristic area in terms of graphical expression.

More specifically, the dynamic range control parameter calculator 512calculates the photoelectric conversion characteristic setting valuecapable of setting the current photoelectric conversion characteristic701 having the inflection point 703 to the new photoelectric conversioncharacteristic 702 having the inflection point 704 whose output level ishigher than that of the inflection point 703 by increasing the sensoroutput corresponding to the inflection point of the photoelectricconversion characteristic from Vth1 to Vth2, namely, by handling theluminance Lt1 in a fixed manner. The calculated photoelectric conversioncharacteristic setting value is outputted to the dynamic range controlsignal generator 521, which, in turn, generates a predetermined drivesignal. Thereby, the sensor output Vt1 corresponding to the luminanceLt1, which has been obtained from the logarithmic characteristic area ofthe photoelectric conversion characteristic 701, is obtained from thelinear characteristic area of the photoelectric conversioncharacteristic 702, whereby the target output Vtarget at the point 706of the linear characteristic area is obtained. As the photoelectricconversion characteristic is shifted from the photoelectric conversioncharacteristic 701 to the photoelectric conversion characteristic 702,the luminance corresponding to the sensor output Vmax is changed, namelydecreased from Lm2 to Lm1, and consequently, the dynamic range isdecreased.

In the dynamic range control, it is impossible to control the imagesensor 30 in such a manner that an image sensing operation is conductedin a relatively high output level area of the linear characteristic areaif the subject luminance for exposure setting is located in a relativelylow output level area of the linear characteristic area, whereas such acontrol is possible in the exposure amount control. This is becausecharacteristic trajectories in the linear characteristic areas of thephotoelectric conversion characteristics 701 and 702 do not change evenif the inflection point is changed. In view of this, it is preferable toconduct the exposure amount control in combination with the dynamicrange control to obtain high contrast.

In the aforementioned dynamic range control parameter calculating step,the photoelectric conversion characteristic is not changed or moved ifthe photoelectric conversion characteristic at the time of acquiring theAE evaluation values has a feature that the target output Vtargetcorresponding to the luminance for exposure setting is obtained from thelinear characteristic area. However, in such a case, even if theexposure time setting value and the aperture setting value take the samevalues as those when the AE evaluation values were obtained last time,the exposure time setting value and the aperture setting value may becalculated this time.

In this way, by AE control through the exposure amount control (A)and/or the dynamic range control (B), it is possible to capture the mainsubject image with an appropriate luminance for exposure amount settingin the linear characteristic area of the photoelectric conversioncharacteristic and to attain a predetermined sensor output.

(Detailed Description on Exposure Amount Control Parameter CalculatingProcess)

In this section, calculation of an exposure amount control parameter,namely, an exposure time setting value and an aperture setting value tobe implemented by the exposure amount control parameter calculator 511based on the AE evaluation values detected by the evaluation valuedetector 405 is described in detail in the case where the exposureamount control shown in FIG. 18A is conducted.

FIG. 20 is an illustration showing an example of a calculation processfor matching the sensor output corresponding to the luminance Lt1 inFIG. 18A, namely, the luminance for exposure amount setting, with thetarget output Vtarget. Referring to FIG. 20, the curve denoted by thereference numeral α1 represents a photoelectric conversioncharacteristic at the time of acquiring the AE evaluation values. Thephotoelectric conversion characteristic α1 is divided into a linearcharacteristic area 622 and a logarithmic characteristic area 623, withthe inflection point 621, namely, the sensor output Vth, serving as aboundary. The curve denoted by the reference numeral 131 represents aphotoelectric conversion characteristic comprised of a linearcharacteristic area in its entirety, which is obtained by converting thelogarithmic characteristic area 623 in the photoelectric conversioncharacteristic α1 to a linear characteristic area 624.

In FIG. 20, the luminance LtLin at the point A is an average luminancein the linear characteristic area 622 of the photoelectric conversioncharacteristic α1, with its sensor output VtLin corresponding to theluminance LtLin. The luminance LtLog at the point B is an averageluminance in the logarithmic characteristic area 623 of thephotoelectric conversion characteristic α1, with its sensor output VtLogcorresponding to the luminance LtLog. First, data conversion isconducted, so that the point B corresponding to the luminance LtLog inthe logarithmic characteristic area of the photoelectric conversioncharacteristic α1 is shifted to the point C in the linear characteristicarea 624, namely, the sensor output VtLog corresponding to the luminanceLtLog in the logarithmic characteristic area 623 coincides with thesensor output VtLogLin in the linear characteristic area 624. Byimplementing this data conversion, all the data in the photoelectricconversion characteristic α1 can be handled as data in the linearcharacteristic area 624. The data conversion from the logarithmiccharacteristic area 623 of the photoelectric conversion characteristicα1 to the linear characteristic area 624 of the photoelectric conversioncharacteristic β1 is performed with use of a lookup table stored in theLUT storage 518. Then, the sensor output VtAve at the point D iscalculated by implementing the following equation, with use of thesensor output VtLin at the point A and the sensor output VtLogLin at thepoint C. The luminance LtAve corresponding to the sensor output VtAvecorresponds to the luminance Lt1 for exposure amount setting in FIG.18A, namely, the subject luminance for exposure setting.VtAve=(VtLin·k1)+(VtLogLin−(1−k1))

where k1=m/(m+n),

m: total number of pixels used in calculation of luminance LtLin atpoint A, and

n: total number of pixels used in calculation of luminance LtLog atpoint B.

In this way, the sensor outputs VtLin and VtlogLin are calculated basedon the luminances LtLin and LtLog, and the sensor output VtAve iscalculated based on the sensor outputs VtLin and VtLogLin.

Next, the following computation is implemented to obtain a gain (Gain)of the exposure amount for making the sensor output VtAve coincidentwith the target output Vtarget shown in FIG. 18A, a gain Gt of theexposure time and a gain Gs of the aperture value based on the exposureamount gain (Gain), an exposure time T2 based on the exposure time gainGt, and an aperture area S2 based on the aperture gain Gs. Calculationof the gains Gt and Gs with use of the respective equations is performedaccording to the flowchart shown in FIG. 21.Gain=Vtarget/VtAveGt·Gs=Gain

<Equations for Calculating the Exposure Time Gain>Tmax/T1=Gtmax (maximum gain of the exposure time)Tmin/T1=Gtmin (minimum gain of the exposure time)Gain/Gtmax=GGtmax (gain for compensating for insufficiency at themaximum gain)Gain/Gtmin=GGtmin (gain for compensating for insufficiency at theminimum gain)T2=T1·Gt

where T1: exposure time at the time of detecting the AE evaluationvalues,

T2: exposure time after the AE correction,

Tmax: maximum exposure time of the image sensor 30, and

Tmin: minimum exposure time of the image sensor 30.

<Equations for Calculating the Aperture Gain>Smax/S1=Gsmax (maximum gain of the aperture value)Smin/S1=Gsmin (minimum gain of the aperture value)Gain/Gsmax=GGsmax (gain for compensating for insufficiency at themaximum gain)Gain/Gsmin=GGsmin (gain for compensating for insufficiency at theminimum gain)S2=S1·Gs

where S1: aperture area at the time of detecting the AE evaluationvalues,

S2: aperture area after the AE correction,

Smax: maximum aperture ratio of the diaphragm 22, and

Smin: minimum aperture ratio of the diaphragm 22.

As shown in FIG. 21, if VtAve is equal to Vtarget, namely, if theexposure amount gain (Gain)=1.0, and there is no need of exposure amountcontrol, namely, no change of the exposure amount control parameter (YESin Step S211), the exposure time gain Gt=1.0, and the aperture gainGs=1.0 (Step 212). Accordingly, the exposure time and the aperture areaare not changed. On the other hand, if Gain 1.0 (NO in Step S211), andGain>1.0 (YES in Step S213), and Gain≦Gtmax (NO in Step S214), namely,if the exposure amount gain (Gain) is larger than 1.0, exposure amountcontrol is necessary, and the exposure amount gain (Gain) can be handledby the exposure time gain Gt, namely, Gt≦Gtmax, then, Gt=Gain, andGs=1.0 (Step S215).

If Gain≦1.0 (NO in Step S213), and Gain≧Gtmin (NO in Step S216),similarly to Step S215, the exposure amount gain (Gain) is smaller than1.0, exposure amount control is necessary, and the exposure amount gain(Gain) can be handled by the exposure time gain Gt, namely, Gt≧Gtmin,then, Gt=Gain, and Gs=1.0 (Step S217).

If Gain>Gtmax (YES in Step S214), and Gsmax>GGtmax (YES in Step S218),then Gt=Gtmax, and Gs=GGtmax (Step S219). In Step S219, the exposureamount gain (Gain) is larger than the maximum exposure time gain Gtmax,and it is impossible to handle the exposure amount gain (Gain) by theexposure time gain Gt without changing the aperture gain Gs (=1.0).Accordingly, insufficiency of the exposure time gain Gt with respect tothe exposure amount gain (Gain) is handled, namely, compensated for bychanging the aperture gain Gs. The gain GGtmax for compensating forinsufficiency at the maximum exposure time gain Gtmax is used as theaperture gain Gs. The gain GGtmax is used in the above control becausethe gain GGtmax is smaller than the maximum aperture gain Gsmax, namely,there is no need of using the maximum aperture gain Gsmax. Thus, thereis no need of calculating a gain for controlling the diaphragm 22 byimplementing the equation relating to the aperture value.

If Gain<Gtmin (YES in Step S216), and Gsmin<GGtmin (YES in Step S221),then, Gt=Gtmin, and Gs=GGtmin (Step S222). In this case, similarly toStep S219, the exposure amount gain (Gain) is smaller than the minimumexposure time gain Gtmin, and accordingly, it is impossible to handlethe exposure amount gain (Gain) by the exposure time gain Gt withoutchanging the aperture gain Gs (=1.0). Accordingly, insufficiency of theexposure time gain Gt with respect to the exposure amount gain (Gain) ishandled, namely, compensated for by changing the aperture gain Gs. Thegain GGtmin for compensating for insufficiency at the minimum exposuretime gain Gtmin is used as the aperture gain Gs. The gain GGtmin is usedin the above control because the gain GGtmin is smaller than the minimumaperture gain Gsmin, namely, there is no need of using the minimumaperture gain Gsmin. In this arrangement, similar to the above case,there is no need of calculating a gain for controlling the diaphragm 22by implementing the equation relating to the aperture value.

If Gsmax≦GGtmax (NO in Step S218), Gt=Gtmax, and Gs=Gsmax (Step S220).Further, if Gsmin≧GGtmin (NO in Step S221), Gt=Gtmin, and Gs=Gsmin (StepS223). If GGtmax≧Gsmax in Step S220, the maximum aperture gain Gsmax isused as the aperture gain Gs. Similarly, if GGtmin≦Gsmin in Step S223,the minimum aperture gain Gsmin is used as the aperture gain Gs.

In this embodiment, as shown in the flowchart of FIG. 21, the exposuretime gain Gt is prioritized in selecting a control parameter forobtaining the exposure amount gain (Gain), namely, exposure time controlis prioritized. Alternatively, the aperture gain Gs may be prioritized,namely, aperture control may be prioritized. Further, in thisembodiment, the exposure time gain Gt and the aperture gain Gs arecalculated by using a single luminance Lt1 for exposure amount setting.Alternatively, two or more luminances for exposure amount setting may beused for the calculation. In such an altered arrangement, the averagevalue of the exposure time gains Gt, and the average value of theaperture gains Gs, or the maximum/minimum value of the exposure timegains Gt, and the maximum/minimum value of the aperture gains Gs may beused.

Thus, the exposure time gain Gt and the aperture gain Gs are calculated,and the exposure time T2 after the AE correction, and the aperture areaS2 after the AE correction are calculated based on the exposure timegain Gt and the aperture gain Gs, respectively. Then, the exposure timesetting value for the image sensor 30 or for the shutter 23, and theaperture setting value for the diaphragm 22 are calculated based on theexposure time T2, and the aperture area S2, respectively, by dataconversion with use of the corresponding lookup tables. The exposuretime setting value and the aperture setting value obtained by the dataconversion are stored in the photoelectric conversion informationstorage 516. Alternatively, the exposure time setting value and theaperture setting value obtained when the AE evaluation values wereacquired last time may be renewed by the newly obtained correspondingsetting values. The same idea is applicable to the photoelectricconversion characteristic setting value, which will be described in thefollowing.

The shutter control signal generator 523 and the aperture control signalgenerator 525 generate control signals to the shutter driver 61 and theaperture driver 63, which make it possible to set the exposure time,namely, the integration time of the image sensor 30 or the shutter 23 tothe exposure time T2, and to set the aperture area of the diaphragm 22to the aperture area S2, based on the exposure time setting value andthe aperture setting value calculated in the exposure amount controlparameter calculator 511, respectively.

Next, described is an exemplified process for calculating the sensoroutput VtLin corresponding to the linear characteristic area averageluminance LtLin, and the sensor output VtLog corresponding to thelogarithmic characteristic area average luminance LtLog shown in FIG.20. First, a process for calculating the sensor output level VtLincorresponding to the linear characteristic area average luminance LtLinis described. An average luminance (called as “block linear averageluminance”) in the linear characteristic area with respect to each ofthe detection blocks A through AJ in the main subject image area 331shown in FIG. 15 is calculated based on the subject luminanceinformation detected from each of the detection blocks A through AJ. Theblock linear average luminance is calculated with use of an average(called as “color linear average”) in the linear characteristic areawith respect to each of the color components R, G, and B. Specifically,color linear averages of the color component R which have been obtainedfrom the blocks A through AJ are calculated as AveRA, AveRB, . . . , andAveRAJ, respectively. Similarly, color linear averages of the colorcomponent G which have been obtained from the blocks A through AJ arecalculated as AveGA, AveGB, . . . , and AveGAJ, respectively, and colorlinear averages of the color component B which have been obtained fromthe blocks A through AJ are calculated as AveBA, AveBB, . . . , andAveBAJ, respectively. Then, a block linear average luminance withrespect to each of the blocks A through AJ is calculated in accordancewith the following color space conversion equation, with use of thecolor linear averages of the respective color components R, G, and B.For instance, a block linear average luminance AveYA is obtained byimplementing the following equation where AveYA represents a blocklinear average luminance in the detection block A.AveYA=AveRA·K1+AveGA·K2+AveBA·K3

where K1, K2, and K3 are coefficients used in color space conversionfrom RGB to YCbCr, and K1=0.2989, K2=0.5866, and K3=0.1145,respectively.

Calculation is implemented with respect to the detections blocks Bthrough AJ in a similar manner as the calculation is implemented withrespect to the detection block A. Thus, block linear average luminancesAveYA, AveYB, . . . , and AveYAJ are calculated in the respective blocksA through AJ. Then, an average luminance with respect to the entirety ofthe block linear average luminances AveYA, AveYB, . . . , and AveYAJ iscalculated, and set as MainY. The average luminance MainY is the sensoroutput VtLin corresponding to the linear characteristic area averageluminance LtLin.

On the other hand, the sensor output VtLog corresponding to thelogarithmic characteristic area average luminance LtLog is calculated ina similar manner as calculating the sensor output VtLin. Specifically,an average luminance (called as “block log average luminance”) in thelogarithmic characteristic area with respect to each of the detectionblocks A through AJ in the main subject image area 331 shown in FIG. 15is calculated based on the subject luminance information detected fromeach of the detection blocks A through AJ. The block log averageluminance is calculated with use of an average (called as “color logaverage”) in the logarithmic characteristic area with respect to each ofthe color components R, G, and B. Specifically, color log averages ofthe color component R which have been obtained from the blocks A throughAJ are calculated as AveRLogA, AveRLogB, . . . , and AveRLogAJ,respectively. Similarly, color log averages of the color component Gwhich have been obtained from the blocks A through AJ are calculated asAveGLogA, AveGLogB, . . . , and AveGLogAJ, respectively, and color logaverages of the color component B which have been obtained from theblocks A through AJ are calculated as AveBLogA, AveBLogB, . . . , andAveBLogAJ, respectively.

The color log averages in the logarithmic characteristic area withrespect to the respective color components R, G, and B are temporarilyconverted to linear data by data conversion to corresponding values inthe linear characteristic area with use of a lookup table stored in theLUT storage 518. Similarly to the above, the block log averageluminances AveYLogA, AveYLogB, . . . , and AveYLogAJ in the respectivedetection blocks A through AJ are calculated in accordance with thecolor space conversion equation in a similar manner as mentioned above,with use of the linear data. Then, an average luminance with respect tothe entirety of the block log average luminances AveYlogA, AveYLogB, . .. , and AveYLogAJ is calculated, and set as MainYLog. The averageluminance MainYLog is the sensor output VtLogLin corresponding to thelogarithmic characteristic area average luminance LtLog. Alternatively,the color linear averages or the color log averages in the respectivedetection blocks A through AJ with respect to the respective colorcomponents R, G, and B may be obtained by calculating a luminancehistogram in the linear characteristic area or in the logarithmiccharacteristic area with respect to each of the detection blocks Athrough AJ, applying the Gaussian pruning to the luminance histograms,and averaging the respective luminances after the Gaussian pruning.

The following is a modified process for calculating the Gain as theexposure amount control parameter, which has been described referring toFIGS. 20 and 21. First, a maximum luminance (called as “color maximumvalue”) in each of the detection blocks A through AJ in the main subjectimage area 331 is calculated with respect to each of the colorcomponents R, G, and B. Specifically, the color maximum values of thecolor component R in the detection blocks A through AJ are calculated asMaxRA, MaxRB, . . . , and MaxRAJ, respectively. Similarly, the colormaximum values of the color component G in the detection blocks Athrough AJ are calculated as MaxGA, MaxGB, . . . , and MaxGAJ,respectively, and the color maximum values of the color component B inthe detection blocks A through AJ are calculated as MaxBA, MaxBB, . . ., and MaxBAJ, respectively. Then, a maximum block luminance in each ofthe detection blocks A through AJ is calculated in accordance with thefollowing color space conversion equation with use of the maximumluminance with respect to each of the color components R, G, and B. Forinstance, a maximum block luminance MaxYA is obtained by implementingthe following equation where MaxYA represents a maximum block luminancein the detection block A.MaxYA=MaxRA·K1+MaxGA·K2+MaxYA·K3

where K1=0.2989, K2=0.5866, and K3=0.1145, respectively, which is thesame as in obtaining the block linear average luminance.

Calculation is implemented with respect to the detections blocks Bthrough AJ in a similar manner as the calculation is implemented withrespect to the detection block A. Thus, maximum block luminances MaxYA,MaxYB, . . . , and MaxYAJ are calculated in the respective blocks Athrough AJ. Then, a maximum luminance with respect to the entirety ofthe maximum block luminances MaxYA, MaxYB, . . . , and MaxYAJ iscalculated, and set as MaxY. The maximum luminance MaxY is a maximumluminance in the main subject image area 331. The maximum luminance MaxYis the sensor output VtAve2 corresponding to the luminance Ltmax shownin FIG. 22.

Similarly, a minimum luminance (called as “color minimum value”) in eachof the detection blocks A through AJ is calculated with respect to eachof the color components R, G, and B, and the calculated color minimumvalues are set as MinRA, MinRB, . . . , and MinRAJ in case of the colorcomponent R; MinGA, MinGB, . . . , and MinGAJ in case of the colorcomponent G; and MinBA, MinBB, . . . , and MinBAJ in case of the colorcomponent B. Then, minimum block luminances MinYA, MinYB, . . . , andMinYAJ are calculated with respect to the respective detection blocks Athrough AJ in accordance with the color space conversion equations in asimilar manner as mentioned above, with use of the minimum luminanceswith respect to the color components R, G, and B. Subsequently, aminimum value (called as “minimum luminance in the main subject imagearea 331”) with respect to the entirety of the minimum block luminancesMinYA, MinYB, . . . , and MinYAJ is calculated, and set as MinY. Theminimum luminance MinY is the sensor output VtAve1 corresponding to theluminance Ltmin shown in FIG. 22.

A color space conversion is conducted after the color maximum values andthe color minimum values located in the logarithmic characteristic areaare similarly converted into values in the linear characteristic areausing a lookup table. Alternatively, the color maximum values or thecolor minimum values in the respective detection blocks A through AJwith respect to the respective color components R, G, and B may beobtained by calculating a luminance histogram with respect to each ofthe detection blocks A through AJ, applying the Gaussian pruning to theluminance histograms, and performing a predetermined computation withuse of the luminances after the Gaussian pruning.

Alternatively, as shown in FIG. 22, it is possible to calculate a gain,namely, Vtarget1/VtAve1 (called as “first gain”), so that the sensoroutput VtAve1 corresponding to the luminance Ltmin attains apredetermined target output Vtarget1, to calculate a gain, namely,Vtarget2/VtAve2 (called as “second gain”), so that the sensor outputVtAve2 corresponding to the luminance Ltmax attains a predeterminedtarget output Vtarget2, to select a smaller gain between the first gainand the second gain, and to execute the flowchart in FIG. 21 forcalculating the exposure time gain Gt and the aperture gain Gs.

Further alternatively, it is possible to select a larger gain betweenthe first gain and the second gain, or to perform computation with useof the first gain or the second gain exclusively, in place of comparingthe first gain with the second gain for selection. Furtheralternatively, it is possible to use an average of the first gain andthe second gain.

Alternatively, it is possible to calculate the minimum luminance MinYand the maximum luminance MaxY based on the entire luminance histogramwith respect to the entirety of the detection blocks A through AJ, whichis obtained by integrating the luminance histograms with respect to thedetection blocks A through AJ. In such an altered arrangement, theluminance range in the entire luminance histogram is calculated byapplying the Gaussian pruning in a similar manner as mentioned above,and the minimum luminance MinY and the maximum luminance MaxY arecalculated based on the luminance range. Alternatively, it is possibleto obtain one of the maximum and minimum luminances based on theluminance and the luminance range of the other one of the maximum andminimum luminances. For instance, the minimum luminance MinY=maximumluminance MaxY−luminance range, and the maximum luminance MaxY=minimumluminance MinY+luminance range.

(Detailed Description on Dynamic Range Control Parameter CalculatingProcess)

Next, a process for calculating a dynamic range control parameter,namely, a photoelectric conversion characteristic setting value by thedynamic range control parameter calculator 512 based on the AEevaluation values detected by the evaluation value detector 405 in thedynamic range control as shown in FIG. 19 is described in detail.

FIGS. 23A and 23B are illustrations each for explaining a process forcalculating the position of an inflection point of a photoelectricconversion characteristic after shifting of the photoelectric conversioncharacteristic, wherein FIG. 23A shows a case that the photoelectricconversion characteristic is changed to attain a predetermined sensoroutput corresponding to the luminance Lt1, and FIG. 23B shows a casethat the photoelectric conversion characteristic is modeled.

Referring to FIG. 23A, the reference numeral α2 denotes a photoelectricconversion characteristic having an inflection point 711 beforeshifting, and the reference numeral β2 denotes a photoelectricconversion characteristic having an inflection point 712 after shifting.The luminance Lt1 is a subject luminance for exposure setting as in thecase of FIG. 19, and Vth1 and Vth2 are sensor outputs corresponding tothe inflection points 711 and 712, respectively. In FIG. 23A, thephotoelectric conversion characteristic is changed, so that the sensoroutput corresponding to the luminance Lt1 is shifted from the sensoroutput Vt1 at the point E on the photoelectric conversion characteristicα2 to the target output Vtarget at the point F on the photoelectricconversion characteristic β2. In this case, the photoelectric conversioncharacteristic α2 is shifted to the photoelectric conversioncharacteristic β2 by a changed amount ΔVth, whereby the sensor outputVth1 at the inflection point 711 becomes the sensor output Vth2 at theinflection point 712.

The sensor output Vth2 is obtained by using the equation:ΔVtarget=Vtarget−Vt1 where ΔVtarget represents a sensor outputdifference between the points E and F. A process for calculating thesensor output Vth2 is described. As shown in FIG. 23B, a linearcharacteristic area and a logarithmic characteristic area in each of thephotoelectric conversion characteristics α2 and β2 are expressed ingraph-based modeling in terms of the following functions, namely,mathematical equations.

Function by Modeling the Linear Characteristic Area:V=K2·L (common for the photoelectric conversion characteristics α2 andβ2)

Function by Modeling the Logarithmic Characteristic Area:V=K1·ln(L)+Wα (in case of the photoelectric conversion characteristicα2)V=K1·ln(L)+Wβ (in case of the photoelectric conversion characteristicβ2)

where K1 and K2 each represents a constant, L represents an incidentluminance to the image sensor 30 along the axis of abscissas in FIG.23B, and Wα and Wβ each represents an intercept.

Since the sensor output difference ΔVtarget is represented byΔVtarget=Wα−Wβ, the equation: V=K1·ln(L)+Wβ is expressed by thefollowing equation:V=K1·ln(L)+Wα+ΔVtargetVth2 is a sensor output at an intersection 713 of the above modelingequation and the modeling equation: V=K2·L. Accordingly, the sensoroutput Vth2 corresponding to the luminance L is calculated by obtainingthe value “L” that satisfies the following simultaneous equation ofthese two equations for calculating the intersection 713 or coordinates,namely, by obtaining the luminance L shown in FIG. 23B.K1·ln(L)+Wα+ΔVtarget=K2·L

In this embodiment, the sensor output Vth2 is set higher than the targetoutput Vtarget corresponding to the subject luminance Lt1 to obtain thetarget output Vtarget from the linear characteristic area. If, however,the calculated output level Vth2 exceeds the saturated output levelVmax, the photoelectric conversion characteristic of the image sensor 30consists of a linear characteristic area without inclusion of alogarithmic characteristic area.

A setting value for the image sensor 30 corresponding to the calculatedsensor output Vth2, namely, a photoelectric conversion characteristicsetting value for changing the photoelectric conversion characteristic,with which the inflection point of the photoelectric conversioncharacteristic is shifted to the point corresponding to the sensoroutput Vth2, is calculated by data conversion for making the sensoroutput Vth1 coincident with the sensor output Vth2 with use of a lookuptable. The photoelectric conversion characteristic setting valuecalculated by the data conversion is stored in the photoelectricconversion characteristic information storage 516. The dynamic rangecontrol signal generator 521 generates a control signal to the timinggenerating circuit 31 to change the position of the inflection point ofthe photoelectric conversion characteristic of the image sensor 30 in amanner as mentioned above based on the photoelectric conversioncharacteristic setting value calculated by the dynamic range controlparameter calculator 512.

The following is an example of a process for calculating the sensoroutput level Vt1 corresponding to the luminance Lt1 for dynamic rangesetting shown in FIGS. 23A and 23B. First, similarly to the calculationof the sensor output level VtLog corresponding to the logarithmiccharacteristic area average luminance LtLog shown in FIG. 20, alogarithmic characteristic area average luminance in the main subjectimage area 331 constituted of the detection blocks A through AJ shown inFIG. 15 is calculated, and a logarithmic characteristic area averageluminance in the peripheral subject image area 332 constituted of thefirst through sixteenth detection blocks is calculated, as in the caseof calculating the logarithmic characteristic area average luminance inthe main subject image area 331. Then, a larger logarithmiccharacteristic area average luminance is selected between thelogarithmic characteristic area average luminances in the main subjectimage area 331 and in the peripheral subject image area 332, and thesensor output level corresponding to the selected logarithmiccharacteristic area average luminance is set to Vt1.

Alternatively, the following arrangement is proposed. A sensor outputcorresponding to the linear characteristic area average luminance, whichis obtained in a similar manner as obtaining the linear characteristicarea average luminance shown in FIG. 20, is calculated in the mainsubject image area 331 and in the peripheral subject image area 332, aswell as the logarithmic characteristic area average luminances. Anentire characteristic area average luminance is calculated with respectto each of the main subject image area 331 and the peripheral subjectimage area 332 by averaging the linear characteristic area averageluminance and the logarithmic characteristic area average luminance ineach of the main subject image area 331 and the peripheral subject imagearea 332. Then, a sensor output corresponding to a larger entirecharacteristic area average luminance between the entire characteristicarea average luminances in the main subject image area 331 and in theperipheral subject image area 332 is set as a sensor outputcorresponding to the luminance Lt1. If the entire characteristic areaaverage luminances are identical to each other in the main subject imagearea 331 and in the peripheral subject image area 332, a sensor outputcorresponding to any of the entire characteristic area averageluminances may be set as a sensor output corresponding to the luminanceLt1. The same idea is applicable to the control mentioned in thefollowing sections.

Alternatively, the sensor output corresponding to the luminance Lt1 maybe obtained from the logarithmic characteristic area average luminancein the main subject image area 331, or from the entire characteristicarea average luminance comprised of the logarithmic characteristic areaaverage luminance and the linear characteristic area average luminancein the main subject image area 331, or may be obtained from thelogarithmic characteristic area average luminance in the peripheralsubject image area 332, or from the entire characteristic area averageluminance comprised of the logarithmic characteristic area averageluminance and the linear characteristic area average luminance in theperipheral subject image area 332.

The following is a modified process for calculating the sensor outputcorresponding to the luminance Lmax. First, similarly to the calculationof the sensor output corresponding to the maximum luminance Ltmax orMaxY as shown in FIG. 22, the maximum luminance in the main subjectimage area 331 is calculated. Likewise, the maximum luminance in theperipheral subject image area 332 is calculated in a similar manner ascalculating the maximum luminance in the main subject image area 331.Then, a larger maximum luminance is selected between the maximumluminances in the main subject image area 331 and in the peripheralsubject image area 332, and the sensor output level corresponding to theselected maximum luminance is obtained, and the luminance correspondingto the sensor output level is set as the luminance Lt1. Alternatively,it is possible to obtain the sensor output corresponding to theluminance Lt1 based on the maximum luminance in the main subject imagearea 331, or to obtain the sensor output corresponding to the luminanceLt1 based on the maximum luminance in the peripheral subject image area332.

As described in the section referring to FIG. 17, the control oflocating the main subject average luminance in the linear characteristicarea is performed based on a judgment of the saturation judging section4055 (see FIG. 14) that the output level of the image sensor 30 is notsaturated. Alternatively, the above control may be performedirrespective of a judgment as to whether saturation is detected by thesaturation judging section 4055, in light of esteeming image quality ofthe main subject image. However, it is possible to temporarily changethe photoelectric conversion characteristic or the integration time toprevent prompt saturation in shooting a subject image which may causesaturation by performing control of locating the main subject averageluminance in the linear characteristic area.

(Step S3-1) Setting of Exposure Amount Control Parameter:

After calculation of the exposure amount control parameter for AEcontrol according to the process described in Step S2-1, AE control isexecuted based on the exposure amount control parameter to performactual shooting. In case of shooting a still image, actual shooting isperformed after performing AE control based on the AE evaluation valuesacquired from an image captured by preliminary shooting. In case ofshooting a moving image, the image sensing is performed successivelyafter performing the AE control based on the AE evaluation valuesacquired from an image captured immediately before.

Specifically, the exposure amount control parameter calculated by theexposure amount control parameter calculator 511 of the main controller50 is outputted to the control signal generating section 520. Uponreceiving the exposure amount control parameter, the respective elementsin the control signal generating section 520 generate control signalsfor operating the timing generating circuit 30 and the driving section60, which in turn, generate drive signals for causing the respectiveelements to perform actual exposure amount control operation. Morespecifically, the sensor exposure time control signal generator 522 ofthe control signal generating section 520 generates a control signal tothe image sensor 30, so that a predetermined exposure time can besecured in accordance with the exposure amount control parameter, andsends the control signal to the timing generating circuit 31. Thecontrol signal is, for instance, a signal, which makes it possible toset the time ΔS for making the signal φVPS for the image sensor 30 toattain a middle potential M in the timing chart shown in FIG. 9 to anappropriate value based on the exposure amount control parameter,namely, a signal that makes it possible to optimize the integration timefrom the timing t1 when the resetting of the parasitic capacitance ofthe photodiode PD is ended to the timing t2 when the readout of thevideo signal of the next frame image is started. The timing generatingcircuit 31 generates a timing signal for controlling the exposure timeof the image sensor 30 in accordance with the inputted control signal todrive the image sensor 30.

Similarly to the sensor exposure time control signal generator 522, theshutter control signal generator 523 generates a control signal forcontrolling a shutter speed or a shutter opening time of the shutter 23according to the exposure time based on the exposure amount controlparameter. The control signal is sent to the shutter driver 61 of thedriving section 60. Upon receiving the control signal, the shutterdriver 61 generates a drive signal for driving the shutter 23 based onthe control signal to cause the shutter 23 to perform a shutter openingoperation according to the exposure amount control parameter.

Similarly to the shutter control signal generator 523, the aperturecontrol signal generator 525 generates a control signal for setting theaperture area of the diaphragm 22 based on the exposure amount controlparameter. The control signal is sent to the aperture driver 63. Uponreceiving the control signal, the aperture driver 63 generates a drivesignal for driving the diaphragm 22 based on the control signal to causethe diaphragm 22 to perform an aperture area setting operation accordingto the exposure amount control parameter.

As mentioned above, there are three kinds of controls as the exposureamount control, i.e., the integration time control, specifically,driving control of the image sensor 30 by the timing generating circuit31, shutter speed control, and aperture control. All the three controlsmay be performed altogether. However, it is desirable to prioritize theelectronic-circuitry-based exposure amount control by the timinggenerating circuit 31, as described referring to the flowchart shown inFIG. 21, in the point of achieving the control in a short time.

(Step S3-2) Setting of Dynamic Range Control Parameter:

In case of performing AE control according to the embodiment of theinvention by dynamic range control, the dynamic range control parametercalculated in the dynamic range control parameter calculator 512 of themain controller 50 is outputted to the control signal generating section520. Upon receiving the dynamic image control parameter, the controlsignal generator 521 generates a control signal for causing the relevantelements to perform actual dynamic range control operation.

Specifically, the dynamic range control signal generator 521 generates acontrol signal to the image sensor 30 to control the switching point ofthe output level, namely, the inflection point, for switching thephotoelectric conversion characteristic from a linear characteristicarea to a logarithmic characteristic area depending on the photoelectricconversion characteristic setting value of the image sensor 30 which hasbeen calculated in the dynamic range control parameter calculator 512,and sends the control signal to the timing generating circuit 31. Thecontrol signal is, for instance, a control signal, based on which thesignal φVPS for the image sensor 30 in the timing chart shown in FIG. 9is desirably set in accordance with the calculated dynamic range controlparameter.

More specifically, in view of the fact that the inflection point ischanged by controlling the intensity of the voltage VPH or the durationof the time ΔT in the signal φVPS, the dynamic range control signalgenerator 521 generates a control signal for controlling the signal φVPSbased on the dynamic range control parameter, and sends the controlsignal to the timing generating circuit 31. Upon receiving the controlsignal, the timing generating circuit 31 generates a timing signal tocontrol the dynamic range of the image sensor 30 to drive the imagesensor 30 in a predetermined photoelectric conversion characteristicstate.

As mentioned above, the operation of the digital camera 1 has beendescribed by primarily focusing on AE control. In the actual digitalcamera 1, AF control and WB control are performed as well as AE control.Similarly to AE control, AF control can be performed based on the AFevaluation values acquired from the image captured by the image sensor30. For instance, it is possible to calculate the AF evaluation value bya so-called “hill-climbing search technique”, wherein luminancehistograms obtained from the detection blocks O, P, U, and V of the mainsubject image area 331 shown in FIG. 14 are utilized, and a peak pointat which the contrast to the luminance at an adjacent point is maximumis detected by the evaluation value detector 405. In this case, it isdesirable to detect the AF evaluation value from each of the linearcharacteristic area and the logarithmic characteristic area of the imagesensor 30 so as to utilize the features of the respective characteristicareas. For instance, it is desirable to use the AF evaluation valueobtained from the logarithmic characteristic area for rough metering inAF control, and to use the AF evaluation value obtained from the linearcharacteristic area for precise metering.

The AF evaluation values detected by the evaluation value detector 405are sent to the AF control parameter calculator 513 of the maincontroller 50. The AF control parameter calculator 513 calculates an AFcontrol parameter corresponding to the AF evaluation value, and sendsthe AF control parameter to the zoom/focus control signal generator 524.Upon receiving the AF control parameter, the zoom/focus control signalgenerator 524 generates a control signal corresponding to the inputtedAF control parameter, and sends the control signal to the zoom/focusdriver 62. Upon receiving the control signal, the zoom/focus driver 62generates a drive signal corresponding to the control signal to drivethe lens group 21 in the lens barrel 20 for focusing based on the drivesignal.

Similarly to AF control, WB control can be performed based on the WBevaluation values acquired from the image captured by the image sensor30. In this case, it is desirable to detect the WB evaluation valuesfrom each of the linear characteristic area and the logarithmiccharacteristic area of the image sensor 30, as in the case of the AFevaluation values. Specifically, the WB evaluation values are detectedfrom neutral images based on the captured image. It is desirable toprepare two kinds of images as the neutral images, wherein one isobtained from the linear characteristic area, and the other is obtainedfrom the logarithmic characteristic area, and to detect R, G, B levels,namely, R-Log, G-Log, B-Log, R-Lin, G-Lin, and B-Lin based on the twoimages. The evaluation value detector 405 detects the WB evaluationvalues, and sends the WB evaluation values to the white balancecontroller 406, which, in turn, performs WB correction to achieveoptimal color balance.

In the digital camera 1 performing the aforementioned AE controlaccording to the embodiment of the invention, an image signal of atarget subject is constantly acquired from the linear characteristicarea of the image sensor 30, and also, a predetermined dynamic range issecured by utilizing the logarithmic characteristic area. For instance,even if a subject luminance is low, an image signal having a highcontrast is obtained from the linear characteristic area, and a dynamicrange having a high luminance area is secured from the logarithmiccharacteristic area. Thus, this arrangement enables to cause the imagesensor to attain an optimal sensing output and video output commensuratewith the amount of a subject light image.

By implementing the aforementioned signal processing, the processedimage is temporarily stored in an image memory or in the memory card412, or displayed on the LCD section 106 as a monitor image. In thisway, the sensing operation by the digital camera 1 is completed.

Second Embodiment

In the following a second embodiment of the invention is describedreferring to FIGS. 24 through 29.

The second embodiment is different from the first embodiment in the AEcontrolling method. The AE control in the first embodiment is performedby the exposure amount control as shown in FIGS. 18A and 18B, and by thedynamic range control as shown in FIG. 19. The exposure amount control(A) is performed based on control of the exposure time, namely, theintegration time of the image sensor 30, or the opening time of theshutter 23, and based on control of the aperture value, namely, theaperture area of the diaphragm 22, and the dynamic range control (B) isperformed based on control of the photoelectric conversioncharacteristic, namely, the position of the inflection point, assummarized below.

(A) Exposure amount control based on exposure time control, i.e.,control of the integration time or the shutter opening time, or based onaperture control, i.e., control of the aperture area of the diaphragm;and

(B) Dynamic range control based on control of the photoelectricconversion characteristic, i.e., control of the position of theinflection point On the other hand, in AE control of the secondembodiment, as summarized in [A] through [C], exposure amount control isperformed in such a manner that exposure amount control based onaperture control, and exposure amount control based on control of theintegration time or the shutter opening time are performed independentlyof each other, and dynamic range control is performed by controlling thephotoelectric conversion characteristic, namely, the position of theinflection point, as in the case of the first embodiment.

[A] Exposure amount control based on aperture control, namely, controlof the aperture area of the diaphragm;

[B] Dynamic range control based on control of the photoelectricconversion characteristic, namely, control of the position of theinflection point; and

[C] Exposure amount control based on exposure time control, namely,control of the integration time or the shutter opening time

The second embodiment is different from the first embodiment in the AEcontrolling method, specifically, as shown in FIG. 24, in that a digitalcamera 1 a as the second embodiment has an image sensor 30 a, a maincontroller 50 a comprised of a calculating section 510 a and a controlsignal generating section 520 a, and a timing generating circuit 31 a.Elements in the second embodiment that are equivalent or identical tothose in the first embodiment are denoted by the same referencenumerals, and description thereof is omitted herein.

First, the image sensor 30 a is described in detail. FIG. 25 is anexemplified circuitry arrangement of each pixel in the image sensor 30 aof the digital camera 1 a as the second embodiment. The pixels in theimage sensor 30 a correspond to the pixels G11 through Gmn in the firstembodiment, as shown in FIG. 7. As shown in FIG. 25, each of the pixelsG11 through Gmn of the image sensor 30 a comprises a photodiode PD1,transistors T10 through T13 each comprised of a metal oxidesemiconductor field effect transistor (MOSFET), and a floating diffusion(FD). An n-channel MOSFET is adopted as the transistors T10 through T13.The symbols VDD, φRSB, φRST, φTX, and φV represent signals or voltagesto the respective transistors T10 through T13, and GND represents theground.

The photodiode PD1 is a light sensing section or a photoelectricconversion section, and outputs an electrical signal or a photocurrentIPD1 commensurate with the amount of incident light from a subject. Thetransistor T12 and each of constant current sources corresponding to theconstant current sources shown in FIG. 7 constitute an amplifyingcircuit, which is a source follower circuit i.e. a source followeramplifier to amplify a voltage V1OUT, which will be described later,namely, to conduct current amplification. The transistor T13 is atransistor for reading out a signal, and serves as a switch which isturned on and off in response to a voltage or a signal φV applied to thegate thereof. Specifically, the source of the transistor T13 isconnected to an output signal line unit 306 corresponding to the outputsignal line unit 306 shown in FIG. 7, and the electric current which hasbeen amplified by the transistor T12 is drawn to the output signal lineunit 306, as an output current when the transistor T13 is turned on.

The transistor T10 is operated as a switch which is turned on and off inresponse to a voltage applied to the gate thereof, and functions as aso-called transfer gate, which performs switching oftransferring/non-transferring the photocurrent IPD1 or the electriccharge generated in the photodiode PD1 to the FD in response to turningon and off, namely, high and low of the gate potential of the transistorT10. The photocurrent PID1 generated in the photodiode PD1 flows to theparasitic capacitance of the photodiode PD1 for accumulating electriccharge to thereby generate a voltage in accordance with the accumulatedelectric charge. If the transistor T10 is in an ON state at this time,the electric charge i.e. a negative charge accumulated in the parasiticcapacitance is transferred to the FD. The FD is a charge holder whichtemporarily holds the electric charge i.e. a signal charge, and servesas a capacitor which converts the charge into a voltage, namely,performs charge-voltage conversion.

The transistor T11 i.e. a reset gate transistor performs switching ofapplying/non-applying a reset bias to the FD in response to turning onand off, namely, high and low of the gate voltage of the transistor T11.For instance, if the transistor T11 is in an ON state, the transistorT10 is also kept in an ON state, and accordingly, a reset bias isapplied between φRSB and GND, with the transistor T11, the FD, thetransistor T10, and the photodiode PD1 being interposed therebetween.Also, by setting the gate voltage to a middle potential (called as “Midpotential” or “Mid”), which will be described later, a linear conversionand a log conversion are implemented, respectively, by charge-voltageconversion by the FD and the transistor T11 of converting the chargewhich is transferred from the photodiode PD1 to the FD, namely, acurrent flowing in the FD, into a voltage.

Specifically, a reset current corresponding to the Mid potential flowsin the transistor T11 to thereby cause the source of the transistor T11to attain a potential corresponding to the reset current. If thepotential attained by the charge transferred from the photodiode PD1 tothe FD is smaller than the source potential of the transistor T11corresponding to the Mid potential, namely, if the subject luminance foran image sensing is low, i.e., the subject image is dark, and the amountof light to be incident onto the photodiode PD1 is small, charge-voltageconversion as the linear conversion is performed by the FD. On the otherhand, if the potential attained by the charge transferred from thephotodiode PD1 to the FD exceeds the source potential of the transistorT11, namely, the subject luminance for the image sensing is high i.e.the subject image is bright, and the amount of light to be incident ontothe photodiode PD1 is large, charge-voltage conversion as the logconversion is performed by the transistor T11.

By performing the above operations, a voltage as a linear outputobtained by integration of the photocurrent IPD1 in the FD, or a voltageas a log output obtained by current-voltage conversion in accordancewith the photocurrent IPD1 in the transistor T11 is generated at theconnection node of the FD and the transistor T12, namely, at the outputV1OUT. Specifically, the output value in the linear characteristic areaof the photoelectric conversion characteristic is an integrated value ofthe photocurrent IPD1 in the FD. However, in a certain area of thelogarithmic characteristic area where the potential by the chargeaccumulated in the FD exceeds the source current of the transistor T11i.e. a reset gate, a current equivalent to the photocurrent IPD1 flowsin the transistor T11, and a voltage obtained by current-voltageconversion of the photocurrent IPD1 in the transistor T11 is generatedas the output value in the FD. The current-voltage conversion in thetransistor T11 corresponds to the log conversion. Therefore, as will bedescribed later, the output value in the linear characteristic area hasan integration effect in the FD and in the parasitic capacitance, andthe gradient of the linear characteristic area is changed in accordancewith the exposure time of the image sensor 30 a. On the other hand, theoutput value in the logarithmic characteristic area does not have anintegration effect in the FD or in the parasitic capacitance. Therefore,the photoelectric conversion characteristic in the logarithmiccharacteristic area is fixed or unchanged independently of the exposuretime of the image sensor 30 a. In other words, there is no time factorin the logarithmic characteristic area.

Subsequently, in response to turning on of the transistor T13, anamplified current corresponding to the respective voltages is drawn fromthe transistor T12 to the output signal line unit 306 via the transistorT13, as an output current. Thus, in the second embodiment, the imagesensor 30 a i.e. each pixel has a circuitry arrangement provided withthe FD, namely, a transfer gate and a reset gate to the FD, unlike theimage sensor 30 which has the integration circuit comprised of thecapacitor C and the transistor T3. This arrangement enables to obtain anoutput signal which is acquired by linear conversion or log conversioncommensurate with the subject luminance or the incident luminance to theimage sensor 30 a.

As compared with the image sensor 30, the image sensor 30 a has asimplified circuitry arrangement. For instance, the image sensor 30 ahas a circuitry arrangement excluding the integration circuit comprisedof the capacitor C and the transistor T3. This arrangement enables toincrease the aperture ratio to the image sensor 30 a to thereby improvesensitivity or sensor output to the light incident onto the imagesensing plane of the image sensor 30 a.

FIGS. 26A and 26B are examples of timing charts relating to an imagesensing operation of each pixel in the image sensor 30 a shown in FIG.25. FIG. 26A is a timing chart relating to a charge accumulation or anexposing operation of all the pixels in a vertical blank period, andFIG. 26B is a timing chart relating to a charge sweeping operation ofpixels in each row by vertical scanning in a horizontal blank periodafter termination of the charge accumulation. In this embodiment, inlight of polarities of the n-channel MOSFET, the transistor is turned onwhen the respective signals are set high (Hi), and turned off when therespective signals are set low (Low).

First, referring to FIG. 26A, the signal φRST is set Hi at the timingindicated by the arrow 801, and the signal φTX is set Hi at the timingindicated by the arrow 802. Thereby, the reset bias is applied to theFD. Then, in response to temporary setting of the signal φRSB to Low atthe timing indicated by the arrow 803 during a period when both of thesignals φRST and φTX are set Hi, the potential of the FD is brought to astate where high luminance light is allowed to be incident, namely, astate where electric charge accumulation is started from 0. Then, aresetting operation or a refresh operation to the FD is performed byreturning or setting the potential of the FD to Hi at the timingindicated by the arrow 804 with respect to all the pixels. By conductingthe resetting operation to the FD, the electric charge in the FD isstabilized.

Thereafter, the signal φRST to the gate of the transistor T11 is changedfrom Hi to Mid with respect to all the pixels simultaneously at thetiming indicated by the arrow 805 to perform electric chargeaccumulation in the FD, and voltage conversion of the accumulated chargeby the linear conversion or the log conversion in the period when thesignal φRST is set to Mid, namely, during a time period from the timingindicated by the arrow 805 to the timing indicated by the arrow 806. Atthis time, a ratio of the linear conversion or the log conversion ischanged according to a difference ΔRST between Hi and Mid of the signalφRST.

Specifically, as the potential difference ΔRST is increased, the sensoroutput level Vth at the inflection point of the photoelectric conversioncharacteristic of the image sensor 30 a is increased, namely, the ratioof the linear characteristic area in the photoelectric conversioncharacteristic is increased, in other words, the ratio of thelogarithmic characteristic area is decreased. On the other hand, as thepotential difference ΔRST is decreased, the sensor output level Vth atthe inflection point of the photoelectric conversion characteristic ofthe image sensor 30 a is decreased, namely, the ratio of the linearcharacteristic area in the photoelectric conversion characteristic isdecreased, in other words, the ratio of the logarithmic characteristicarea is increased. Thus, the output level at the inflection point of thephotoelectric conversion characteristic of the image sensor 30 a,namely, the position of the inflection point is controlled bycontrolling the potential difference ΔRST, as explained in the sectionreferring to FIG. 11. This control corresponds to the control shown inFIG. 29, which will be described later.

The control of the potential difference ΔRST, namely, the Mid level, canbe regarded as a control of changing the position of the inflectionpoint, while keeping the gradients in the linear characteristic area andthe logarithmic characteristic area unchanged, namely, a control ofchanging the switching point or offset from the linear characteristic tothe log characteristic.

Referring back to FIG. 26A, in a state that the signal φTX is set Hi,the period from the timing when the signal φRST is changed from Hi toMid to the timing when the signal φTX is changed from Hi to Low, and thesignal φRST is changed from Mid to Low, namely, a period during whichthe signal φRST is set Mid, is set as a time ΔL. The time ΔL correspondsto an exposure time or an integration time of each pixel in the imagesensor 30 a, namely, an electric charge accumulation time, i.e., anaccumulation time ΔL. Control of the duration of the time ΔL isperformed by a sensor exposure time control signal generator 522 a viathe timing generating circuit 31 a, which will be described later.Control of the magnitude of the ΔRST, namely, control of the potentiallevel of the signal φRST from Hi to Mid is performed by a dynamic rangecontrol signal generator 521 a via the timing generating circuit 31 a,which will be described later.

Subsequently, the charge which has been accumulated in the FD for thecharge accumulation time ΔL is held, namely, the charge accumulation isterminated with respect to all the pixels simultaneously by setting thesignal φTX from Hi to Low at the timing indicated by the arrow 806, andby setting the signal φRST from Mid to Low at the timing indicated bythe arrow 807.

Next, referring to FIG. 26B, after the charge accumulation with respectto all the pixels is terminated in FIG. 26A, the transistor T13 isturned on in response to setting of the signal φV for a pixel in acertain row selected by vertical scanning of a vertical scanning circuitcorresponding to the vertical scanning circuit 301 shown in FIG. 7 to Hiat the timing indicated by the arrow 811. Thereby, the chargeaccumulated in the FD of each pixel in the selected row is read out orswept to a corresponding vertical signal line corresponding to theoutput signal lines 306-1, 306-2, . . . , and 306-m shown in FIG. 7. Thecharge readout operation at the timing indicated by the arrow 811 isstarted in response to termination of the charge accumulation at thetiming indicated by the arrow 806 or 807 shown in FIG. 26A. The readoutsignal charge is transferred along the vertical signal line fortemporary holding in each sample hold circuit of a selection circuit 308corresponding to the selection circuit 308 shown in FIG. 7.

Thereafter, the signal φRST for each pixel in the selected row ischanged from Low to Hi at the timing indicated by the arrow 812, andsimultaneously, the signal φRSB for each pixel in the selected row ischanged from Hi to Low at the timing indicated by the arrow 813 to setthe potential of the FD to a state where high luminance light is allowedto be incident. Then, the signal φRSB is returned or set Hi at thetiming indicated by the arrow 814 while keeping the signal φRST Hi, andthe FD is reset to a value corresponding to the threshold value of thereset gate or the transistor T11. In response to setting the signal φVHi at the timing indicated by the arrow 815 in this state, a noisesignal is read out to the vertical signal line, and is held in a noisesample hold circuit provided in a correction circuit 309 correspondingto the correction circuit 309 shown in FIG. 7, which will be describedlater.

The signals φSHS and φSHN shown in FIG. 26B are sample hold controlsignals in a sample hold circuit for signal in the selection circuit308, and in the sample hold circuit for noise in the correction circuit309, respectively, which will be described later. As indicated by thereference numerals 816 and 817, during periods respectively indicated bythe reference numerals 811 and 815 when the signal φV is set Hi, asignal i.e. an image signal, and a noise i.e. a noise signal aresample-held based on the sample hold control signals in the respectivesample hold circuits of the selection circuit 308 and the correctioncircuit 309, as shown by “SIGNAL S/H” and “NOISE S/H” in FIG. 26B. Then,an image signal whose threshold variation of the reset gate has beenremoved is acquired with respect to each pixel in each row bycalculating a difference between the image signal and the noise signalgenerated by the sample holding operations, namely, by subtracting thenoise signal from the image signal. The signal φTX is constantly set Lowduring a horizontal blank period.

Referring back to FIG. 24, similarly to the first embodiment, an AEcontrol parameter calculating unit 5110 a in the main controller 50 a ofthe digital camera 1 a calculates a control parameter for setting theoptimal exposure amount and the photoelectric conversion characteristicor the dynamic range of the image sensor 30 a for an image sensingoperation so as to perform exposure control i.e. AE control commensuratewith a subject luminance. In the second embodiment, control parametersregarding the respective controls [A], [B], and [C] are calculated. TheAE control parameter calculating unit 5110 a includes an exposure amountcontrol parameter calculator 511 a comprised of an exposure time controlparameter calculator 5111 and an aperture control parameter calculator5112, and a dynamic range control parameter calculator 512 a.

The exposure time control parameter calculator 5111 calculates a controlparameter for optimizing the exposure time, and calculates an exposuretime setting value according to a subject luminance, based on the AEevaluation values detected by an evaluation value detector 405, andphotoelectric conversion characteristic information of the image sensor30 a obtained at the time of acquiring the AE evaluation values storedin a photoelectric conversion characteristic information storage 516.The exposure time setting value is a value for controlling the exposuretime or the integration time of the image sensor 30 a, or the openingtime of a shutter 23 to set an exposure amount, based on which thephotoelectric conversion characteristic is changed to obtain apredetermined sensor output corresponding to a predetermined luminancefor exposure amount setting.

The aperture control parameter calculator 5112 calculates a controlparameter for optimizing the aperture value. Similarly to the exposuretime setting value, the aperture control parameter calculator 5112calculates an aperture setting value according to a subject luminancebased on the AE evaluation values detected by the evaluation valuedetector 405, and the photoelectric conversion characteristicinformation of the image sensor 30 a obtained at the time of acquiringthe AE evaluation values stored in the photoelectric conversioncharacteristic information storage 516. The aperture setting value is avalue for controlling the aperture value, namely, the aperture area ofthe diaphragm 22 to set an exposure amount based on which thephotoelectric conversion characteristic is changed to obtain apredetermined sensor output corresponding to a predetermined luminancefor exposure amount setting.

The dynamic range control parameter calculator 512 a calculates acontrol parameter for optimizing the photoelectric conversioncharacteristic, namely, the dynamic range of the image sensor 30 aaccording to the subject luminance. The dynamic range control parametercalculator 512 a calculates a photoelectric conversion characteristicsetting value for controlling the position of the inflection point ofthe photoelectric conversion characteristic, so that the image sensor 30a acquires the photoelectric conversion characteristic i.e. the dynamicrange, based on which a saturated output level corresponding to thesubject luminance for dynamic range setting is obtained. Thephotoelectric conversion characteristic information of the image sensor30 a obtained at the time of acquiring the AE evaluation values storedin the photoelectric conversion characteristic information storage 516is referred to in calculating the photoelectric conversioncharacteristic setting value.

The exposure amount control parameter, namely, the exposure time controlparameter and the aperture control parameter, and the dynamic rangecontrol parameter respectively calculated by the exposure amount controlparameter calculator 511 a and the dynamic range control parametercalculator 512 a of the main controller 50 a are outputted to a controlsignal generating section 520 a. Upon receiving the control parameters,respective elements in the control signal generating section 520 agenerate control signals for operating the timing generating circuit 31a and a driving section 60, which, in turn, generate drive signals forcausing the relevant elements to perform actual exposure amount controloperation. The control signal generating section 520 a in the secondembodiment is different from the control signal generating section 520in the first embodiment in that a dynamic range control signal generator521 a and a sensor exposure time control signal generator 522 a areoperated to drive the image sensor 30 a in the second embodiment.

Specifically, the dynamic range control signal generator 521 a generatesa drive signal to the timing generating circuit 31 a, namely, to theimage sensor 30 a for controlling the output level at the inflectionpoint at which the photoelectric conversion characteristic is switchedfrom a linear characteristic area to a logarithmic characteristic areabased on the photoelectric conversion characteristic setting value ofthe image sensor 30 a calculated by the dynamic range control parametercalculator 512 a, and sends the drive signal to the timing generatingcircuit 31 a. As mentioned above, the inflection point is changed bycontrolling the photoelectric conversion characteristic of the imagesensor 30 a by the potential difference ΔRST of the signal φRST for theimage sensor 30 a between Hi and Mid. Thus, the dynamic range of theimage sensor 30 a is controlled in accordance with the subject luminanceby controlling the drive signal to the timing generating circuit 31 a soas to control the magnitude of the difference ΔRST or the Mid level ofthe signal φRST. The timing generating circuit 31 a generates a timingsignal for controlling the dynamic range of the image sensor 30 a basedon a drive signal corresponding to the inputted difference ΔRST to drivethe image sensor 30 a.

The sensor exposure time control signal generator 522 a generates adrive signal to the timing generating circuit 31 a for securing anecessary exposure time based on the exposure time setting valuecalculated by the exposure time control parameter calculator 511, andsends the drive signal to the timing generating circuit 31 a. Asmentioned above, the drive signal is a control signal for optimizing theaccumulation time ΔL, with which the signal φRST for the image sensor 30a is set to the middle potential Mid based on the exposure time settingvalue. The timing generating circuit 31 a generates a timing signal forcontrolling the exposure time of the image sensor 30 a based on thedrive signal corresponding to the inputted accumulation time ΔL in asimilar manner as mentioned above, and drives the image sensor 30 a.

The aperture setting value calculated by the aperture control parametercalculator 5112 is outputted to the aperture control signal generator525, which, in turn, generates a drive signal to the driving section 60for setting the aperture area of the diaphragm 22 based on the aperturesetting value, and sends the drive signal to the driving section 60.Similarly to the aperture control signal generator 525, the shuttercontrol signal generator 523 generates a control signal for setting theshutter speed of the shutter 23 in accordance with the exposure timebased on the exposure time setting value calculated by the exposure timecontrol parameter calculator 5111, and sends the control signal to thedriving section 60.

Next, the aperture-control-based exposure amount control [A], thephotoelectric-conversion-characteristic-based dynamic range control [B],and the exposure-time-control-based exposure amount control [C] in theAE control according to the second embodiment are described referring toFIGS. 27 through 29.

FIG. 27 is a graph showing how the photoelectric conversioncharacteristic of the image sensor 30 a is changed in case of performingthe control [A]. Unlike the first embodiment in which the axes ofabscissas in the photoelectric conversion characteristic graphs (seeFIGS. 18A and 18B) represent “incident luminance to the image sensor”,the axis of abscissas in FIG. 27 represents “subject luminance”. This isthe same for the graphs shown in FIGS. 28 and 29. In the secondembodiment, the luminance in the axis of abscissas is a subjectluminance itself, namely, an absolute luminance, in place of a sensorinput luminance obtained as a result of exposure of the image sensor toa subject light image for a certain time, namely, an integration time ora shutter opening time, which includes a time factor. The axis ofordinate in the second embodiment represents a sensor output, as in thecase of the first embodiment.

As shown in FIG. 27, the entirety of the photoelectric conversioncharacteristic of the image sensor 30 a is changed or shifted in one ofthe directions shown by the arrows H by controlling the aperture value,namely, the aperture area of the diaphragm 22. In this case, thephotoelectric conversion characteristic is shifted in the leftwarddirection of the arrows H from a photoelectric conversion characteristic821 to a photoelectric conversion characteristic 831 by increasing theaperture area of the diaphragm 22. On the other hand, the photoelectricconversion characteristic is shifted in the rightward direction of thearrows H by decreasing the aperture area of the diaphragm 22.Specifically, similarly to the controls shown in FIGS. 18A and 18B inthe first embodiment, for instance, calculated is the photoelectricconversion characteristic 831 in such a manner that the sensor outputcorresponding to a predetermined luminance Lt1 for exposure amountsetting in the linear characteristic area of the photoelectricconversion characteristic 821 becomes a target output Vtarget, namely,the sensor output level corresponding to the luminance Lt1 is increasedfrom the point 822 to the point 823. In this case, the inflection point824 is shifted in parallel to the inflection point 825, and the sensoroutput Vth does not change. In other words, the photoelectric conversioncharacteristic is changed in such a manner that the subject luminancefor obtaining the target output Vtarget is changed or lowered from Lt2to Lt1. It is possible to change the sensor output obtained in relationto a subject luminance as an absolute luminance by changing the aperturearea of the diaphragm 22 independently of the exposure time, i.e., freeof the exposure time, by handling the subject luminance as mentionedabove. Thus, the AE control can be performed in such a manner as tochange the entirety of the photoelectric conversion characteristic, asshown in FIG. 27.

The aperture area of the diaphragm 22 necessary for changing thephotoelectric conversion characteristic in such a manner that the sensoroutput corresponding to the luminance Lt1 for exposure amount settingbecomes Vtarget is calculated as in the case of the first embodiment.Specifically, as explained in the section referring to FIGS. 20 and 22,an exposure amount gain (Gain) is calculated by implementing theequation: Gain=Vtarget/VtAve, an exposure time gain Gt and an aperturegain Gs are calculated based on the Gain by implementing the equation:Gain=Gt·Gs according to the flowchart shown in FIG. 21, and an exposuretime T2 and an aperture area S2 are calculated based on the exposuretime gain Gt and the aperture gain Gs. The aperture area S2 obtained bythe calculation corresponds to the aperture area used in the exposureamount control in FIG. 27, and the setting value for obtaining theaperture area corresponds to the aperture setting value.

The aperture control parameter calculator 5112 in the calculatingsection 510 a calculates the aperture area of the diaphragm 22. Theexposure time is automatically calculated in the course of calculatingthe aperture area. Alternatively, the exposure time control parametercalculator 5111 may perform interim computation, namely, calculation ofthe exposure time gain Gt and the aperture gain Gs in the course ofcalculating the aperture area, and then, the aperture control parametercalculator 5112 may calculate the aperture area based on the calculatedaperture gain Gs.

Similarly to the first embodiment, it is possible to obtain a mainsubject luminance and a peripheral subject luminance by implementingmulti-pattern metering by a multi-pattern metering section 4051 toobtain information such as a luminance histogram, and a maximum/minimumluminance by a histogram calculator 4052 and a maximum/minimum luminancecalculator 4054, respectively, and to calculate luminance informationsuch as the luminance Lt1 to be used in the above calculation based onthe information. The same operation is performed in the controls [B] and[C] as in the control [A].

FIG. 28 is a graph showing how the photoelectric conversioncharacteristic of the image sensor 30 a is changed in case of performingthe control [C]. In the first embodiment, the linear characteristic areaand the logarithmic characteristic area, namely, the entirety of thephotoelectric conversion characteristic is changed by controlling theexposure time (see FIG. 18A). In the second embodiment, the sensoroutput obtained by the log conversion with respect to the subjectluminance does not change even if the exposure time or the integrationtime is changed in light of the property of the image sensor 30 a.Namely, there is no time factor regarding the exposure time in thelogarithmic characteristic area in the second embodiment. Accordingly,in the second embodiment, the gradient of the linear characteristic areais changed without changing the logarithmic characteristic area, namely,without changing the position and the gradient of the logarithmiccharacteristic area, by changing the exposure time.

Referring to FIG. 28, the gradient K1 in the linear characteristic areaof a photoelectric conversion characteristic 841 shown by the solid lineis changed to the gradient K2 in the linear characteristic area of aphotoelectric conversion characteristic 851 shown by thetwo-dotted-chain line by changing the exposure time, and vice versa. Inthis case, the linear characteristic area in the photoelectricconversion characteristic is moved leftward, namely, the gradient of thelinear characteristic area is shifted from K1 to K2 by increasing theexposure time. On the other hand, the linear characteristic area of thephotoelectric conversion characteristic is moved rightward by decreasingthe exposure time. In this way, the gradient of the linearcharacteristic area is changed, but the logarithmic characteristic areais unchanged. Accordingly, the photoelectric conversion characteristicis shifted from the photoelectric conversion characteristic 841 to thephotoelectric conversion characteristic 851 in such a manner that theposition of the inflection point of the linear characteristic area andthe logarithmic characteristic area is apparently shifted from theinflection point 842 to the inflection point 852, for instance, in theobliquely downward direction corresponding to one of the arrows I alongthe gradient of the logarithmic characteristic area. The ratio of thelinear characteristic area to the logarithmic characteristic area in thephotoelectric conversion characteristic is changed by changing thegradient of the linear characteristic area, namely, by apparentlyshifting the position of the inflection point.

Specifically, for instance, the sensor output corresponding to theluminance Lt100 is increased from the sensor output level at the point843 in the photoelectric conversion characteristic 841 to the sensoroutput level Vtarget at the point 853 in the photoelectric conversioncharacteristic 851 by controlling the exposure time. The sensor outputis increased according to increase of the exposure time, even if thesubject luminance is the same. In other words, the photoelectricconversion characteristic is changed in such a manner that the subjectluminance for obtaining the target output Vtarget is changed or loweredfrom the luminance Lt200 in the photoelectric conversion characteristic841 to the luminance Lt100 in the photoelectric conversioncharacteristic 851.

The exposure time control is performed by control of the integrationtime, namely, the accumulation time ΔL of the image sensor 30 a and/orthe opening time i.e. the shutter speed of the shutter 23. Theintegration time of the image sensor 30 a or the opening time of theshutter 23 necessary for changing the photoelectric conversioncharacteristic such that the sensor output corresponding to theluminance Lt100 for exposure amount setting becomes the target outputVtarget is calculated as in the case of the control [A]. Specifically,an exposure amount gain (Gain) is calculated by implementing theequation: Gain=Vtarget/VtAve, an exposure time gain Gt and an aperturegain Gs are calculated based on the Gain by implementing the equation:Gain=Gt·Gs, and an exposure time T2 and an aperture area S2 arecalculated based on the exposure time gain Gt and the aperture gain Gs.The exposure time T2 obtained by the calculation corresponds to theexposure time to be used in the exposure amount control in FIG. 28, andthe setting value for obtaining the exposure time corresponds to theexposure time setting value.

The exposure time control parameter calculator 5111 calculates theexposure time setting value. The aperture area is automaticallycalculated in the course of calculating the exposure time setting value.Alternatively, the aperture control parameter calculator 5112 mayperform interim computation, namely, calculation of the exposure timegain Gt and the aperture gain Gs in the course of calculating theexposure time, and then, the exposure time control parameter calculator5111 may calculate the exposure time based on the calculated exposuretime gain Gt.

In this way, it is possible to change the linear characteristic areawhile keeping the dynamic range unchanged, namely, without changing thesubject luminance Lm100 corresponding to the sensor output saturationlevel Vmax in FIG. 28, by performing exposure amount control based onwhich the linear characteristic area is changed while keeping thelogarithmic characteristic area unchanged through control of theexposure time.

In other words, the control [C] makes it possible to implement thefollowing control. For instance, if a user wishes to increase theexposure amount in the entirety of a subject image, namely, to obtain abright image from an image captured in a dark place, the control [A] isperformed (see FIG. 27), wherein the entirety of the photoelectricconversion characteristic is changed to increase the dynamic range.Further, for instance, if the user wishes to control the exposure amountsuch as contrast or brightness in an image area of a low luminance,although a satisfactory dynamic range is secured, the control [C] isexecuted. In this way, the arrangement provides high latitude inselecting the exposure amount controlling method, and consequently,enables to perform the AE control with high precision. Alternatively,the digital camera 1 a may have an arrangement of performing the control[A], or an arrangement of performing the control [C], in other words,the exposure amount control may be performed by either one of theaperture control and the exposure time control.

FIG. 29 is a graph showing how the photoelectric conversioncharacteristic of the image sensor 30 a is changed in case of performingthe control [B]. In this control, the dynamic range is controlled bycontrolling the photoelectric conversion characteristic, namely, thesensor output Vth at the inflection point. Specifically, as in the caseof the first embodiment shown in FIG. 19, the position of the inflectionpoint is shifted, for example, from the inflection point 864corresponding to the sensor output Vth1 to the inflection pointcorresponding to the sensor output Vth to decrease the sensor outputcorresponding to the subject luminance Lm20 for dynamic range settingfrom the sensor output Vover at the point 862 in a photoelectricconversion characteristic 861 to the sensor output Vmax i.e. thesaturated output level of the image sensor 30 a at the point 872 in aphotoelectric conversion characteristic 871, in other words, to increasethe maximum luminance capable of obtaining the sensor output Vmax fromthe luminance Lm10 at the point 863 to the luminance Lm20 at the point872, namely, to increase the dynamic range. In this case, thelogarithmic characteristic area is shifted in parallel to the areabefore the shifting in one of the directions indicated by the arrows J,namely, in the sensor output axis direction in a state that the gradientdoes not change by changing the inflection point.

In this way, the photoelectric conversion characteristic setting valueregarding control of the position of the inflection point for dynamicrange control is calculated by the dynamic range control parametercalculator 512 a. A process for calculating the photoelectric conversioncharacteristic setting value is the same as described referring to FIG.23 in the first embodiment. Specifically, an inflection point e.g. theinflection point 874 in FIG. 29 corresponding to Vth2 of thephotoelectric conversion characteristic after shifting is calculated tosecure a necessary dynamic range by modeling the linear characteristicarea and the logarithmic characteristic area of the photoelectricconversion characteristic, and a control value for changing thephotoelectric conversion characteristic, based on which the inflectionpoint e.g. the inflection point 864 corresponding to Vth1 of thephotoelectric conversion characteristic before shifting becomes theinflection point after the shifting is calculated as the photoelectricconversion characteristic setting value.

The dynamic range control signal generator 521 a generates a controlsignal for changing the output level Vth at the inflection point basedon the calculated photoelectric conversion characteristic setting value,namely, a signal for controlling the difference ΔRST i.e. the Mid levelof the signal φRST. The output level at the inflection point in thephotoelectric conversion characteristic of the image sensor 30 a iscontrolled by controlling the difference ΔRST. Referring to FIG. 29, ifthe output level at the inflection point 864 in the photoelectricconversion characteristic 861 is shifted to the sensor output 865 by thedifference ΔRST, the sensor output is decreased from the sensor output865 to the sensor output 875 by decreasing the difference ΔRST, forinstance. As a result, the position of the inflection point is shiftedto the inflection point 874 in the photoelectric conversioncharacteristic 871. Conversely, the inflection point is shifted from theinflection point 874 to the inflection point 864 by increasing thedifference ΔRST. In this way, the control [B] is realized by control ofthe difference ΔRST.

Similarly to the first embodiment, in the second embodiment, it ispossible to implement control of the photoelectric conversioncharacteristic i.e. the position of the inflection point based on thephotoelectric conversion characteristic setting value if a saturationjudging section 4055 (see FIG. 14) judges that the output level of theimage sensor 30 a is not saturated, and to lower the output level at theinflection point by the changed amount ΔVth based on the saturated pixelnumber if the saturation judging section 4055 judges that the outputlevel of the image sensor 30 a is saturated (see FIG. 23A).Specifically, it is possible to change the photoelectric conversioncharacteristic in such a manner as to increase the dynamic range, sothat an image is captured at a higher luminance side.

In the AE control by the respective controls [A], [B], and [C] in thesecond embodiment, similarly to the first embodiment, the AE control isperformed to obtain the sensor output corresponding to a predeterminedsubject luminance for exposure setting from the linear characteristicarea of the image sensor, namely, to sense the subject luminance forexposure setting in the linear characteristic area. In case of thecontrol [A], if the subject luminance for exposure setting i.e. for theexposure amount setting is Lt1, for instance in FIG. 27, the control [A]is performed to obtain the photoelectric conversion characteristic 831,so that the target output Vtarget corresponding to the luminance Lt1 isobtained from the linear characteristic area.

In the above case, similarly to the first embodiment, if the sensoroutput corresponding to the luminance Lt1 is located in the logarithmiccharacteristic area before the control [A] is performed, namely, in FIG.27, if the sensor output corresponding to the luminance Lt1 is locatedin the logarithmic characteristic area of the photoelectric conversioncharacteristic 831, exposure amount control i.e. aperture control ofdecreasing the aperture value, namely, decreasing the aperture area isconducted to obtain the photoelectric conversion characteristic, so thatthe sensor output corresponding to the luminance Lt1 is located in thelinear characteristic area, namely, the entirety of the photoelectricconversion characteristic 831 is shifted in the rightward direction ofthe arrows H.

Exposure amount control of increasing the aperture area is performed toobtain the photoelectric conversion characteristic, so that the sensoroutput is located in a relatively high output level area e.g. at thepoint 823, namely, the photoelectric conversion characteristic isshifted from the photoelectric conversion characteristic 821 to thephotoelectric conversion characteristic 831, if the sensor outputcorresponding to the luminance Lt1 is located in a relatively low outputlevel area, e.g., the sensor output corresponding to the luminance Lt1is located at such a low level as the point 822 in the linearcharacteristic area. In this way, the control [A] is performed in such amanner that the output of the image sensor 30 a corresponding to thesubject luminance for exposure setting is located in a relatively highoutput level area in the linear characteristic area. This arrangementenables to increase the contrast even from a low luminance subjectimage. The control of changing the photoelectric conversioncharacteristic, so that the target output Vtarget is located in thelinear characteristic area, is executed by the aperture controlparameter calculator 5112 based on the aperture setting value.

In the control [C], referring to FIG. 28, if the subject luminance forexposure setting i.e. for exposure amount setting is Lt100, forinstance, exposure amount control by exposure time control, namely,control of the integration time or control of the shutter opening timeis performed to obtain the photoelectric conversion characteristic 851,so that the target output Vtarget corresponding the luminance Lt100 islocated in the linear characteristic area. In this case, similarly tothe control [A], if the sensor output corresponding to the luminanceLt100 is obtained from the logarithmic characteristic area e.g. thesensor output corresponding to the luminance Lt100 is located in thelogarithmic characteristic area of the photoelectric conversioncharacteristic 851, exposure amount control, namely, exposure timecontrol to shorten the exposure time is performed to locate the sensoroutput corresponding to the luminance Lt100 in the linear characteristicarea, namely, to shift the linear characteristic area rightward bychanging the gradient of the linear characteristic area from K2 to K1,for instance. Further, if the sensor output corresponding to theluminance Lt1 is located in a relatively low output level area of thelinear characteristic area, for instance, if the sensor outputcorresponding to the luminance Lt100 is as low as the point 843 in thelinear characteristic area, exposure amount control of increasing theexposure time is performed to change the photoelectric conversioncharacteristic, so that the sensor output is located in a relativelyhigh output level area e.g. at the point 853, namely, the photoelectricconversion characteristic is changed from the photoelectric conversioncharacteristic 821 to the photoelectric conversion characteristic 831,namely, to change the gradient of the linear characteristic area from K1to K2. In this way, the control [C] is performed in such a manner that asensor output corresponding to the subject luminance for exposuresetting is located in a relatively high output level area in the linearcharacteristic area. This arrangement enables to increase the contrasteven from a low luminance subject image. The control of changing thephotoelectric conversion characteristic, so that the target outputVtarget is located in the linear characteristic area is executed by theexposure time control parameter calculator 5111 based on the exposuretime setting value.

In the control [B], referring to FIG. 29, for instance, if the subjectluminance for exposure setting i.e. for dynamic range setting is Lt10,dynamic range control, namely, photoelectric conversion characteristiccontrol of obtaining the photoelectric conversion characteristic isperformed, so that the target output Vtarget corresponding to theluminance Lt10 is located in the linear characteristic area. In thiscase, the photoelectric conversion characteristic is changed from thephotoelectric conversion characteristic 871 to the photoelectricconversion characteristic 861. Specifically, control of changing theposition of the inflection point from the inflection point 874 to theinflection point 864, namely, control of setting the inflection point toa high output level of the image sensor is performed. The control ofchanging the photoelectric conversion characteristic, so that the targetoutput Vtarget is located in the linear characteristic area, is executedby the dynamic range control parameter calculator 512 a based on thephotoelectric conversion characteristic setting value.

In the AE control parameter calculating step in each of the controls[A], [B], and [C], the photoelectric conversion characteristic is notchanged or shifted if the photoelectric conversion characteristicobtained at the time of acquiring the AE evaluation values has a featurethat the target output Vtarget can be already obtained in relation tothe luminance for exposure amount setting as described above. However,in such a case, even if the aperture setting value, the exposure timesetting value, or the photoelectric conversion characteristic settingvalue takes the same value as the corresponding value when the AEevaluation values were obtained last time, the aperture setting value,the exposure time setting value, or the photoelectric conversioncharacteristic setting value may be calculated this time.

In this way, implementing the AE control in such a manner that thecontrols [A], [B], and [C] are executable independently of each other,namely, the aperture-control-based exposure amount control [A] and/orthe exposure-time-control-based exposure amount control [C], and/or thephotoelectric conversion characteristic-control-based dynamic rangecontrol [B] are executable, enables to sense a luminance of a mainsubject image for exposure setting i.e. exposure amount setting in thelinear characteristic area of the photoelectric conversioncharacteristic, while securing a predetermined sensor output level.Alternatively, the AE control may be performed by using one of thecontrols [A], [B], and [C], namely, by singly performing the control[A], [B], or [C], or by combined control of these controls [A], [B], and[C] to sense a luminance for exposure setting in the linearcharacteristic area. For instance, the dynamic range control [B] may beperformed in addition to the aperture-control-based exposure timecontrol [A] and/or the exposure-time-control-based exposure time control[C]. The above arrangements enable to achieve AE control having highlatitude by independent control of the control [A] and/or the control[C], and/or the control [B], and to achieve efficient AE controldepending on the combination of these three controls [A], [B] and [C].

In the foregoing, the preferred embodiments of the invention have beenfully described. The invention is not limited to the above. Forinstance, the following modifications (I) through (VII) are applicable.

(I) In the first embodiment, the p-channel MOSFET is used in each of thepixels of the image sensor 30. Alternatively, an n-channel MOSFET may beused. In the second embodiment, the n-channel MOSFET is used in each ofthe pixels of the image sensor 30 a. Alternatively, a p-channel MOSFETmay be used.

(II) As far as an image sensor can implement the operations as shown inthe image sensors according to the first and second embodiments, anyimage sensor may be used. For instance, in the first embodiment, theentirety of the photoelectric conversion characteristic is changed inrelation to the aperture control and the exposure time control, as theexposure amount control. In the second embodiment, although the entiretyof the photoelectric conversion characteristic is changed in relation tothe aperture control, observing the exposure time control, the linearcharacteristic area has an integration effect, and the photoelectricconversion characteristic in the linear characteristic area is changedin relation to the exposure time, but the logarithmic characteristicarea does not have an integration effect, and the photoelectricconversion characteristic in the logarithmic characteristic area is notchanged irrespective of change of the exposure time. In both of thefirst and second embodiments, however, dynamic range control based oncontrol of the position of the inflection point is performed.

(III) In the first or the second embodiment, the subject luminance isdetected by the image sensor 30 or by the image sensor 30 a.Alternatively, the subject luminance i.e. the AE evaluation values maybe detected with use of a metering device which is providedindependently of the image sensor 30 or the image sensor 30 a, such as adevice for metering the subject luminance according to multi-patternmetering with use of plural light receiving elements. However, it isdesirable to detect the subject luminance or the AE evaluation valuesbased on an image signal obtained from an image actually captured by theimage sensor 30 or by the image sensor 30 a in light of simplifying themechanism of the image sensing apparatus.

(IV) The first (second) embodiment is constructed such that the imagesensing apparatus comprises the shutter 23 and the image sensor 30(image sensor 30 a) to control the exposure time. Alternatively, it ispossible to provide either one of the shutter 23 and the image sensor 30(image sensor 30 a) to control the exposure time.

(V) In the first embodiment, the setting values for the image sensor 30,the shutter 23, and the diaphragm 22, which are obtained by dataconversion with respect to the exposure time T2 and the aperture area S2with use of the lookup tables, are set as the exposure time settingvalue and the aperture setting value, respectively. Alternatively, it ispossible to set the exposure time gain Gt and the aperture gain Gs, orthe exposure time T2 and the aperture area S2, as the exposure timesetting value and the aperture setting value, respectively. Likewise,the output level Vth2 or the changed amount ΔVth at the inflection pointmay be set as the photoelectric conversion characteristic setting value.Similarly to the first embodiment, in the second embodiment, it ispossible to set the exposure time gain Gt and the aperture gain Gs asthe exposure time setting value and the aperture setting value,respectively, and to set the output level at the inflection point as thephotoelectric conversion characteristic setting value.

(VI) In the first and second embodiments, RGB data is used as image datafor calculating evaluation values. Alternatively, it is possible to useimage data other than the RGB data, such as complementary color imagedata and monochromatic image data.

(VII) Dividing the image sensing area 330, namely area dividing may beperformed e.g. by a spot metering system or by a partial metering systemusing a central part, in place of the multi-pattern metering. Furtheralternatively, the block arrangement concerning the image sensing area330 comprised of the main subject image area 331 and the peripheralsubject image area 332 may be other than the arrangement as shown inFIG. 15. Further, in the first and second embodiments, the evaluationvalue is calculated with respect to each of the blocks. Alternatively,it is possible to calculate evaluation values in each of two blocks ofthe image sensing area 330 consisting of the main subject image area 331and the peripheral subject image area 332. Further alternatively, it ispossible to divide the image sensing area 330 into three or more blocks,in place of dividing the image sensing area 330 into the main subjectimage area 331 and the peripheral subject image area 332, to calculateevaluation values based on luminance information in each of the blocks,and to perform AE control based on the evaluation values. As a furtheraltered form, it is possible to calculate a single evaluation value inthe image sensing area 330 without dividing the image sensing area 330into blocks, and to perform AE control based on the single evaluationvalue. Further alternatively, it is possible to arbitrarily set theblocks in the image sensing area 330 as mentioned above in response touser's manipulation/instruction.

Although the present invention has been fully described by way ofexample with reference to the accompanying drawings, it is to beunderstood that various changes and modifications will be apparent tothose skilled in the art. Therefore, unless otherwise such changes andmodifications depart from the scope of the present invention hereinafterdefined, they should be construed as being included therein.

What is claimed is:
 1. An image sensing apparatus comprising: an imagesensor which generates an electrical signal commensurate with an amountof incident light, and has a photoelectric conversion characteristiccomprised of a linear characteristic area, where the electrical signalis outputted after being linearly converted in relation to the amount ofincident light, and a logarithmic characteristic area, where theelectrical signal is outputted after being logarithmically converted inrelation to the amount of incident light; an exposure evaluation valuedetector configured to detect an exposure evaluation value based onluminance information acquired from a subject in sensing an image of thesubject; and an exposure controller configured to acquire a settingvalue for exposure based on the exposure evaluation value detected bythe exposure evaluation value detector to control exposure of the imagesensing apparatus, wherein the exposure controller is configured todetermine a subject luminance for exposure setting based on the exposureevaluation value, and is configured to control the exposure such that anoutput of the image sensor corresponding to the subject luminance forexposure setting is obtained from the linear characteristic area of thephotoelectric conversion characteristic of the image sensor; theexposure controller including: an exposure amount controller configuredto control an exposure amount; and a dynamic range controller configuredto control the photoelectric conversion characteristic of the imagesensor, wherein the dynamic range controller preferentially controls thephotoelectric conversion characteristic; wherein if the subjectluminance for exposure setting is lower than a predetermined value, theexposure amount controller controls the exposure amount in combinationwith the control of the photoelectric conversion characteristic by thedynamic range controller.
 2. The image sensing apparatus according toclaim 1, wherein the exposure evaluation value is detected, by theexposure evaluation value detector, each from a main subject image area,and a peripheral subject image area of an image sensing area of theimage sensor, the image sensing area being comprised at least of themain subject image area, and the peripheral subject image area locatedin a periphery of the main subject image area, and the subject luminancefor exposure setting is selected from the exposure evaluation valuedetected in the main subject image area.
 3. The image sensing apparatusaccording to claim 1, wherein the exposure controller includes aphotoelectric conversion characteristic information storage which storesthe photoelectric conversion characteristic of the image sensor acquiredat the time of detecting the exposure evaluation value by the exposureevaluation value detector.
 4. The image sensing apparatus according toclaim 1, wherein the exposure amount controller includes a photoelectricconversion characteristic information storage which stores thephotoelectric conversion characteristic of the image sensor acquired atthe time of detecting the exposure evaluation value detected by theexposure evaluation value detector, and an exposure amount controlparameter calculator which calculates a control parameter for optimizingthe exposure amount, and the exposure amount control parametercalculator calculates an exposure amount control parameter based on theexposure evaluation value detected by the exposure evaluation valuedetector, and the photoelectric conversion characteristic stored in thephotoelectric conversion characteristic information storage.
 5. Theimage sensing apparatus according to claim 1, wherein the exposureamount is controlled in such a manner that the output of the imagesensor corresponding to the subject luminance for exposure setting isoutputted from a relatively high output level area in the linearcharacteristic area.
 6. The image sensing apparatus according to claim1, wherein the dynamic range controller includes a photoelectricconversion characteristic information storage which stores thephotoelectric conversion characteristic of the image sensor acquired atthe time of detecting the exposure evaluation value by the exposureevaluation value detector, and a dynamic range control parametercalculator which calculates a control parameter for optimizing thephotoelectric conversion characteristic of the image sensor according tothe subject luminance, and the dynamic range control parametercalculator calculates a dynamic range control parameter based on theexposure evaluation value detected by the exposure evaluation valuedetector, and the photoelectric conversion characteristic stored in thephotoelectric conversion characteristic information storage.
 7. Theimage sensing apparatus according to claim 1, wherein the image sensoris configured in such a manner as to execute photoelectric conversion inthe logarithmic characteristic area independently of an exposure time,the image sensing apparatus further comprises a diaphragm, the exposureamount controller includes an aperture controller which controls theexposure amount based on an aperture setting value relating to controlof an aperture area of the diaphragm, and/or an exposure time controllerwhich controls the exposure amount based on an exposure time settingvalue relating to control of the exposure time to the image sensor, andthe exposure amount controller controls the exposure amount by theaperture controller and/or the exposure time controller in such a mannerthat the output of the image sensor corresponding to the subjectluminance for exposure setting is obtained from the linearcharacteristic area of the image sensor, the aperture controller and theexposure time controller being configured to control the exposure amountindependently of each other.
 8. The image sensing apparatus according toclaim 7, wherein the exposure amount is controlled in such a manner thatthe output of the image sensor corresponding to the subject luminancefor exposure setting is obtained from a relatively high output levelarea in the linear characteristic area of the image sensor.
 9. An imagesensing apparatus comprising: an image sensor which generates anelectrical signal commensurate with an amount of incident light, and hasa photoelectric conversion characteristic comprised of a linearcharacteristic area, where the electrical signal is outputted afterbeing linearly converted in relation to the amount of incident light,and a logarithmic characteristic area, where the electrical signal isoutputted after being logarithmically converted in relation to theamount of incident light; an exposure evaluation value detectorconfigured to detect an exposure evaluation value based on luminanceinformation acquired from a subject in sensing an image of the subject;and an exposure controller configured to acquire a setting value forexposure based on the exposure evaluation value detected by the exposureevaluation value detector to control exposure of the image sensingapparatus, the exposure controller including an exposure amountcontroller configured to control an exposure amount and a dynamic rangecontroller configured to control the photoelectric conversioncharacteristic of the image sensor, wherein the exposure controller isconfigured to determine a subject luminance for exposure setting basedon the exposure evaluation value, and is configured to control theexposure by using at least one of the exposure amount controller and thedynamic range controller such that an output of the image sensorcorresponding to the subject luminance for exposure setting is obtainedfrom the linear characteristic area of the photoelectric conversioncharacteristic of the image sensor, wherein the exposure evaluationvalue is detected, by the exposure evaluation value detector, each froma main subject image area, and a peripheral subject image area of animage sensing area of the image sensor, wherein the subject luminancefor exposure setting is selected from the exposure evaluation valuedetected in the main subject image area; and wherein if the subjectluminance for exposure setting is lower than a predetermined value, theexposure amount controller controls the exposure amount in combinationwith the control of the photoelectric conversion characteristic by thedynamic range controller.
 10. An image sensing apparatus comprising: animage sensor which generates an electrical signal commensurate with anamount of incident light, and has a photoelectric conversioncharacteristic comprised of a linear characteristic area, where theelectrical signal is outputted after being linearly converted inrelation to the amount of incident light, and a logarithmiccharacteristic area, where the electrical signal is outputted afterbeing logarithmically converted in relation to the amount of incidentlight: an exposure evaluation value detector configured to detect anexposure evaluation value based on luminance information acquired from asubject in sensing an image of the subject; and an exposure controllerconfigured to acquire a setting value for exposure based on the exposureevaluation value detected by the exposure evaluation value detector tocontrol exposure of the image sensing apparatus, the exposure controllerincluding an exposure amount controller configured to control anexposure amount and a dynamic range controller configured to control thephotoelectric conversion characteristic of the image sensor, wherein theexposure controller is configured to determine a subject luminance forexposure setting based on the exposure evaluation value, and isconfigured to control the exposure by using at least one of the exposureamount controller and the dynamic range controller such that an outputof the image sensor corresponding to the subject luminance for exposuresetting is obtained from the linear characteristic area of thephotoelectric conversion characteristic of the image sensor, wherein theexposure controller further includes a photoelectric conversioncharacteristic information storage configured to store the photoelectricconversion characteristic of the image sensor acquired at the time ofdetecting the exposure evaluation value by the exposure evaluation valuedetector; and wherein if the subject luminance for exposure setting islower than a predetermined value, the exposure amount controllercontrols the exposure amount in combination with the control of thephotoelectric conversion characteristic by the dynamic range controller.11. The image sensing apparatus according to claim 10, wherein theexposure amount controller includes an exposure amount control parametercalculator which calculates a control parameter for optimizing theexposure amount, and the exposure amount control parameter calculatorcalculates an exposure amount control parameter based on the exposureevaluation value detected by the exposure evaluation value detector, andthe photoelectric conversion characteristic stored in the photoelectricconversion characteristic information storage.
 12. The image sensingapparatus according to claim 10, wherein the dynamic range controllerincludes a dynamic range control parameter calculator which calculates acontrol parameter for optimizing the photoelectric conversioncharacteristic of the image sensor according to the subject luminance,and the dynamic range control parameter calculator calculates a dynamicrange control parameter based on the exposure evaluation value detectedby the exposure evaluation value detector, and the photoelectricconversion characteristic stored in the photoelectric conversioncharacteristic information storage.